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,  lOR&VORD 


The  solution  of  a  large  number  of  problems  irtiich  are  of  practical  importance  in 
Tarious  fields  of  radio  engineering  (radio  communications,  radar,  radio  navigation, 
automation  and  remote  control,  electronic  computers  ,  etc.)  is  ijQ)ossible  without  a 
more  thorough  and  expanded  knowledge  of  mathematics  in  the  domain  of  the  theozy  of 
probabilities  in  general  and  in  the  domain  of  the  theozy  of  random  processes  in 
particular. 

There  exist  a  number  of  good  ^stematic  courses  on  probability  theozy.  Bow* 
ever,  in  his  practical  work  an  engineer  lacks,  as  a  rule,  the  opporttmity  to  study 
such  a  course  in  its  entirety.  Furthemore,  the  presentation  of  the  material  in 
such  books,  iMle  sufficiently  sequential  and  strict  in  the  mathematical  sense,  is 
not  aimed  at  solving  concrete  engineering  problems,  and  does  not  contain  the  techni¬ 
cal  exan^les  neeessazy  T  >r  the  radio  engineer,  idiich  could  serve  as  specimens  in 
the  solution  of  a  large  number  of  problems*  to  now,  there  are  no  textbooks  with 
a  systematic  exposition  of  the  mathematical  theozy  of  random  processes,  in  a  fora 
accessible  to  radio  engineers* 

There  exists  a  need  for  a  textbook  in  idiich,  along  with  the  neeessazy  idnlmum 
of  information  on  the  theozy  of  random  variables  and  random  processes,  considerable 
space  would  be  devoted  to  carefully  selected  and  worked-out  radio-engineering* ex- 
aiqples  iMeh  illustrate  the  possibilities  made  available  to  radio  engineers,  in  the 
investigation  of  a  number  of  problems,  by  the  use  of  probability-theory  methods. 

In  attesqpting  to  satisfy  to  a  certain  extent  the  requests  of  radio  specialists 
for  such  a  textbook,  the  author  considers  the  basic  purpose  of  the  present  book  to 
be  such  a  presentation  of  probability-theozy  methods,  applicable  to  radio  engi¬ 
neering,  as  would  assist  the  engineer  in  making  use  of  these  methods  in  his  pro¬ 
fessional  work*  The  material  presented  makes  no  claim  to  completeness  with  respect 
to  mathematical  results,  and  especially  not  with  respect  to  the  physical  treatment 


T 


of  such  results.  In  seveml  pieces  it  is  neeessezy  to  seerifiee  nethenatical  strict, 
ness  in  fesor  of  e  more  accessible  presentation,  the  solitaxy  purpose  of  lAiich  is  to 
proTide  engineers  vith  the  necessaiy  technical  apparatus* 

As  a  result  of  the  United  scope  of  the  book  it  has  also  been  impossible  to  de> 
vote  the  necessaiy  attention  to  the  physical,  treataent  of  results,  ahicb  vere  re. 
garded  merely  as  illustrations  of  aathenatical  nethods. 

The  basis  of  the  present  book  consists  of  a  lecture  course  on  the  theory  of 
random  processes  vhich  has,  during  the  past  several  years,  been  delivered  by  the 
author  to  speeialist.eandidates  in  radio  engineering. 

It  is  assumed  that  besides  the  fundamentals  of  differential  and  integral  calcu. 
lus,  the  reader  has  a  command  of  classical  harmonic  analysis  (series  and  Fourier 
Integral)  within  the  scope  of  the  usual  introductory  sections  indLided  in  many 
monographs  on  radio  engineering*. 

txtensive  use  is  made  of  special  functions,  the  properties  of  some  of  which 
(Bessel  and  Gamma  functions)  are  assumed  to  be  familiar  to  the  re.*ider,  idile  the 
properties  of  the  others  (hyperge<»etrie  functions  and  Hemite  polynomials)  appear 
In  the  ^pendices  to  this  book. 

The  book  has  eleven  chapters.  In  the  beginning  are  formulated  the  rules  for 
the  addition  and  multiplication  of  probabilities,  and  wutplts  of  practical  applica. 
tion  of  these  rules  are  given,  as  well  as  certain  of  their  important  consequences. 

Jn  the  second  and  third  chapters  are  examined  distribution  functions  of  one  random 
variable  and  of  groups  of  random  variables,  their  numerical  characteristics,  and 
also  formulas  for  the  transformation  of  distribution  functions  vith  the  functional 
transfoimations  of  continuous  random  variables.  Particular  attention  is  paid  to  the 
mstbod  of  characteristie  functions,  which  is  effectively  employed  for  determining 
the  distribution  functions  of  a  sum  of  random  variables. 

*  Gf,  for  instance,  I.  S*  Gonorovskiy,  Radiosignaly  i  perekbodayye  yavleniya  v 
radiotsspyaUi  (Radio  Signals  and  Transient  Phenomena  in  Radio  Clreuits),  ffoskva, 
Svyas'itdat  (Oomnranications  Publishing  House),  195^. 


r.TS-98ll/T 


Vi 


The  presentation  of  the  central  llHitlng  theorea  and  of  the  rule  of  large  nn»> 
hers  is  not  restricted  to  only  a  proof  of  the  Halting  relationships,  in  estiaate  is 
given  of  the  rate  of  convergence  of  the  suas  of  independent  randoa  variables  to  the 
zkonul  lav.  there  is  introduced  the  relationship  linking  the  error  and  reliability 
of  aesaureaents  in  a  finite  nuaber  of  experiaents. 

the  fourth  dupter  eoiq>letes  the  round  of  questions  oonneeted  with  the  theory 
of  randoa  variables,  liiich  is  essential  to  an  understanding  of  the  succeeding  six 
eh^ters  Mhich  deal  with  the  theory  of  randoa  processes  and  its  iradio-engineering 
plications,  k  nuaber  of  the  exaaples  used  in  the  first  four  chapters  have  been 
selected  with  a  view  to  the  probleas  exaained  subsequently. 

In  the  fifth  diapter  is  discussed  the  basic  aspects  of  the  theory  of  randoa 
processes:  stea4]r  state  and  ergodicity,  the  correlation  Ametion  sikI  its  properties, 
the  energy  spectruB  of  a  randoa  process  and  its  link  with  the  correlation  function, 
the  derivative  of  a  stationary  randoa  process,  the  average  tmibtr  of  overshoots  of 
the  randoa  process.  There  is  exaained  the  aost  iaportant  fora  of  the  randoa  process 
•  the  noraal  randoa  process,  further  (Chapter  fl)  is  presented  general jaethods  of 
deteraining  statistical  characteristics  with  the  functional  transfomations  of  ran-> 
doa  processes.  The  foUoving  dusters,  ?II  and  vm,  illustrate  non-linear  trans- 
fomations  ty  the  exple  of  the  noraal  randoa  process,  and  Chapter  IZ  illustrates 
the  trar^oraations  of  the  distributim  function  of  this  randoa  process  in  terns  of  an 
electronic  circuit:  the  i-f  aaq>lifier,  the  square-lav  detector,  and  the  video  filter. 

In  Chapter  Z  is  studied  the  energy  speetn  of  pulse-type  randoa  processes,  ar.d 
of  ecntinuous  oscillatioos  nodulated  by  solitude  and  by  phase  (frequency)  by  ran- 
dOB  processes. 

no  elr/enth,  and  concluding  dpter,-  in  ulddi  is  contaiiad  the  eleasnts  of  the 
theory  of  inforaatlon,  has  been  written  with  the  aia  of  drawing  tiie  reader's  atten¬ 
tion  to  this  furiously  developing  brand:  of  the  theory  of  randoa  processes  and  of 
awakening  in  hia  the  desire  to  study  it  in  spedslixed  literature, 

Ihe  literature  on  the  theory  and  applications  of  randoa  processes  is  very  ex- 
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tensive;  it  vas^  thererore,  necessaiY  to  liMt  cursives  to  dting  011I7  a 
imber  of  sources,  lecoieniied  for  a  tore  thorou^  stu4r  of  individiial  qoestion. 
These  refereoces  are  provided  at  the  end  of  eav  chapter* 

'In  order  not  to  r^eat  the  definitiocs  ef  tht:  ^Tdbdls  ttiat  have  been  accepted 
thqr  have  been  listed  at  the  end  of  the  book  in  front  of  the  ^rtOmr, 

The  author  considers  it  his  pleasant  to  eapress  sSnoete  th*"*'*  to  ncefeo 
sor  I*  S.  Gonorovakjjr  and  to  Candidate  E*  T*  Ihrakin,  vfeose  valnaiile  reoaiks  have 
been  taken  into  consideration  in  the  pr^aration  of  this  book  for  the  press* 
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Chi^ter  I 


rOMDAKENTAL  ASPECTS  OP  THE  THEORT  OF  PROBADILITr 

1»  Definltiens  and.  Tenalnology 

For  A  precise  description -of  the  processes  taking  place  in  electrical  or  mechan¬ 
ical  systems  vith  one  or  more  degrees  of  freedom,  it  is  usually  sufficient  to  solve 
a  differential  equation,  or  a  system  of  such  equations  given  the  necessary  number 
of  initial  data  (in  an  electrical  system,  the  voltages,  currents,  or  charges  at  the 
instant  of  time  assumed  as  the  beginning  of  reckoning).  Depending  on  their  complex¬ 
ity,  these  equations  are  solved  Ly  various  mathematical  methods  (for  instance,  in 
the  ease  of  linear  systems,  by  the  operational  method). 

The  difficulty  of  using  the  differential-equation  apparatus  increases  sharply 
in  the  investigation  of  physical  phenomena  with  an  enormous  number  of  degrees  of 
freedom,  Tor  instance  in  the  investigation  of  the  motion  of  interacting 
elementary  particles  of  idilch  substances  are  constituted.  These  difficulties  are 
not  only  computational,  to  a  large  extent  th^  are  linked  vith  the  impossibility, 
in  such  systems,  of  assigning  exact  values  to  the  necessary  quantity  of  initial  data 
(for  Instance,  the  position  and  velocity  of  the  elementary  particles  of  a  substance 
at  a  fixed  moment  in  time).  It  becomes  necessary  to  abandon  an  exact  description  of 
the  behavior  of  a  system  with  a  large  number  of  degrees  of  freedom. 

Hovaver,  in  the  study  of  systems  vith  a  large  number  of  degrees  of  freedom 
there  become  apparent  rules  of  a  special  type,  lAiieh  are  stipulated  by  just  this 
large  number  of  degrees  of  freedom.  The  quantitative  increase  in  the  number  of  de¬ 
grees  of  freedom  leads  to  the  appearance  of  qualitatively  new  rules.  These  are 
called  statistical  rules.  The  study  of  statistical  rules  constitutes  the  subject  of 
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the  theory  of  probabilities 


The  abandonment  of  exact  description  signifies  the  impossibility  of  predicting 
the  results  of  individual  experiments,  idiich  can  vary  from  one  experiment  to  another  | 

I 

in  an  arbitrary  manner.  Such  experiments,  for  which  a  precise  prediction  of  ob-  3 
servatlon  results  in  each  individual  experiment  is  impossible,  are  called  random  | 

ones.  An  example  of  a  random  experiment  is  the  observation  of  noise  in  radio  4 

-I 

circuits.  The  tern  "random**  refers  not  to  the  conditions  of  the  experiment,  idiich  | 

'1 

must  remain  unchanged,  but  to  the  results  of  observation  in  the  multifold  repetition  j 
of  the  same  experiment  (i.e.,  in  the  realisation  of  a  definite  complex  of  eonditions).! 
The  observation  result  may  be  registered  as  the  occurrence  or  non>occurrence  of  a  | 

certain  event  A.  ifhlch  is  called  a  random  event.  Thus,  as  a  result  of  obseirvlng  n 

noise  in  a  radio  receiver,  there  nay  occur  or  not  occur  a  random  event  consisting  of 
the  fact  that  the  amplitude  of  the  noise  will  exceed  the  assigned  voltage  level  U*.  3 

®  .i. 

It  is  impossible  accurately  to  predict  the  result  of  each  individual  random 
experiment.  Bit  if  a  succession  of  a  large  number  of  experiments  is  examined  in  Its  | 
entirety,  an  important  statistical  rule  will  appear:  the  average  results  in  the  ob-  | 
servatlon  of  a  sufficiently  long  series  of  experiments  remain  practically  constant. 
This  constancy  in  the  average  results  of  a  large  number  of  random  experiments  is  of 
a  stable  nature.  4 

Q  random  experiments,  let  the  i*andom  occurence  ^  occur  b  tines.  The  ratio 


n  (1.1) 

Is  called  the  relative  frequency  of  the  occurrence  of  event  A  vith  o  random  experi¬ 
ments.  To  piredict  the  occurrence  and  non-occurrence  of  A  in  un  individual  experi¬ 
ment  Is  impossible.  But  in  observations  of  the  frequency  V  of  the  occurrence  of 
event  A,  as  the  number  n  is  increased  there  is  disclosed  the  property  of  this 
frequency  to  deviate  ever  less  from  a  certain  constant. 

Thus,  for  Instance,  it  is  impossible  to  predict  in  advance  idiether  a  given 
radio-engineering  device  (tube,  unit,  station)  will  operate  without  damage  for  not 
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1«8S  than  T  hours.  However,  in  the  testing  of  a  large  number  of  identical  devices,  | 
the  frequency  of  a  random  event,  which  consists  of  the  fact  that  the  device  has  ^ 

■Si 

operated  not  less  than  1  hours,  will  deviate  ever  less  from  a  certain  constant  as  J 

the  number  of  devices  tested  is  increased.  | 

-  y?: 

Thus  it  is  possible  to  state  the  following  assunption:  for  each  random  event 
Ji  there  is  a  certain  constant  g  to  which  the  frequencies  of  occurrence  v  of  the  eventf 

A  prove  to  be  closer  the  greater  the  number  n  of  random  experiments  <  The  assunption  '4 

-f 

of  the  existence  of  such  a  constant  is  the  experimental  basis  of  probability  theory.  I 

*  i 

This  constant  p  is  called  the  probabilily  of  the  occurrence  of  the  random  event  in  | 

the  realization  of  a  definite  corplex  of  conditions.  .  J 

'"''i 

For  the  frequent  of  occurrence  of  an  event  there  holds  true  the  obvious  in-  | 
equality  0^  V  ^  1.  Since,  according  to  the  definition,  every  probability  p  is  | 

approximately  equal  to  a  certain  frequency  V  ,  it  is  natural  to  consider  that  g  | 

i 

satisfies  the  same  inequality  4 


0  ^  p  ^  1 , 


(1.2) 


An  event  which  occurs  invariably  in  a  random  experiment  is  called  certain.  The 
probability  of  a  certain  event  is  equal  to  1. 

An  event  which  cannot  occur  in  a  random  experiment  is  called  impossible.  The 
probability  of  an  impossible  event  is  equal  to  zero. 

Two  events  are  called  incompatible,  if  they  cannot  both  occur  with  the  reali¬ 
zation  of  a  definite  complex  of  conditions. 

If  it  is  known  for  certain  that  with  every  random  experiment  there  is  observed 
one  of  n  mutually  exclusive  events  A^,  A2i...A„^  then  the  indicated  aggregate  of  n 
events  is  called  a  set. 

Converse  random  events  are  two  incompatible  events  which  constitute  a  set.  The 
simplest  example  of  converse  events  are  certain  and  Impossible  events. 

If  the  probability  of  occurrence  of  one  event  4  depends  on  whether  or  not  an¬ 
other  event  S.  bas  occurred,  then  two  such  events  are  called  dependent  events. 
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Independent  random  events  are  tvo  events  for  \ihlch  the  probability  of  occurence 
of  one  does  not  depend  on  vhether  or  not  the  other  has  taken  place. 

Bouiprobable  events  are  called  random  events  vhich  have  an  equal  probability  of 
occurrence. 

Two  basic  rules  must  be  adhered  to  vith  respect  to  probabilities  of  random 
events:  the  addition  rule  and  the  multiplication  rule,  from  uhlch  a  number  of  im¬ 
portant  consequences  arise.  The  following  paragraphs  are  devoted  to  a  presentation 
of  these  rules  and  to  several  examples  of  their  practical  application. 

2«  The  Addition  Rule. 

If  A2**<*Aq  are  mutually  incompatible  events,  the  probability  of  occurrence 
of  one  of  the  events  A^,  or  A2,...or  Aj^  is  equal  to  the  sum  of  the  probabilities  of 
these  events 


n 

P(A^,  or  or...or  Ajj)  — 


(1.3) 


idiere  ?(k^)  is  the  probability  of  occurrence  of  the  event 

^  If  the  mutually  incompatible  events  A^,  A^^.A^  oonsUtute  a  set,  one  of  these 
events  occurs  for  certain  and,  consequently, 

P(A^,  or  A2.  or  ...  or  A^)  ~  1. 


Considering  (1.3)  ve  obtain 


(1.») 


i.e.,  the  sum  of  the  probabilities  of  occurrence  of  the  random  events  comprising  a 
set  is  equal  to  unity. 

If  a  set  consists  of  two  events  4  and  I,  it  follows  from  formula  (1 .4)  that 


P(4)=I~P0). 


(1.5) 


Since  the  two  events  A  and  I,  comprising  a  set,  are  opposites,  formula(1.5) 
makes  it  possible  to  find  the  probability  of  occurrence  of  event  A  if  the  probability 
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of  the  opposite  event  A  Is  known. 

Let  all  n  random  events  comprising  a  set  be  equlprobable.  l.e., 

*  p  **  const 

for  angr  k  s  1,  2.  ...  d.  Then  according  to  formula  (1.4)  ve  find  the  probability  £ 
of  the  occurrence  of  one  of  n  equlprobable  events,  idiich  comprise  a  set.  to  be 


^p  =  «p=l. 


(1.6) 


The  probability  of  realising  one  of  m  ^  n  events,  idilch.  enter  Into  a  set  of 
n  equlprobable  events.  Is  equal  in  accordance  with  (1.3)  ond  (1.6)  to 


P(A|,  or  A2,or..»or  ijj)  *=  "S'n"’  n  • 


(1.7) 


Suppose,  for  Instance,  that  firing  practice  Is  taking  place  at  a  target  which 
Is  divided  Into  ten  zones*  Suppose  further  the  region  lying  outside  the  target  area 
to  be  worth  zero.  Let  us  denote  by  Aj^  a  random  event  consisting  of  a  hit  on  the 
k»th  zone  of  the  target(ka0,1,2,...10)  with  a  single  shot.  Since  with  a  single 
shot  only  one  zone  of  the  target  is  hit,  any  pair  of  events  ^  and  ^  (k  ^  r)  con¬ 
stitutes  Incompatible  random  events.  The  aggregate  of  all  eleven  events  A^,  A^,  ... 
A|g  obviously  comprises  a  set. 

Let  the  shooter  at  the  target  hit  any  of  Its  zones  (including  the  zero  zone) 
with  equal  probability.  Then  according  to  foraula  (1 .6)  the  probability  of  hitting 
any  of  the  zones  of  the  target  will  be 


The  probability  of  hitting  the  tulls^e.  Into  idilch  enter  five  zones,  fi*om  the 
sixth  to  the  tenth  Inclusive,  Is  In  accordance  with  (1.7)  equal  to 

..-I, 
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Let  another,  aore  experienced  shooter  hit  the  bulls^e  with  a  probability  of 


fa”  It 


Let  tts  ask  the  question,  vhat  is  the  probability  of  hitting  the  buUssye  if 


both  shooters  fire  simultaneously.  Let  us  designate  by  the  event  consisting  of 


a  hit  on  the  bullseye  by  the  first  shooter,  and  by  A2  a  hit  by  the  second  shooter. 


If  the  rule  of  addition  is  used  in  finding  the  probability  of  a  hit  on  the  bullseye,! 


we  obtain 


P(A^  or  Ag)*  + 


This  is  an  absurd  result,  because  the  probability  cannot  exceed  unity.  Ihe 


■Istake  lies  in  the  violation  of  a  condition  of  the  applicability  of  the  addition 


rule,  since  the  tw  events  A^  and  A^,  consisting  in  the  fact  that  each  of  the 


shooters  hits  the  bullseye,  are  compatible,  lb  obtain  a  correct  answer  for  the 


question  iddeh  was  asked,  it  is  necessary  to  make  use  of  the  multiplication  rule. 


J.  The  Multiplication  Buie 


The  probability  of  occurrence  of  two  compatible,  dependent  random  events  A  and 


jg  is  equal  to  the  product  of  the  probability  of  one  of  Uiese  events  by  the  relative 


probability  of  occurrence  of  the  other,  computed  under  the  assumption  that  the 


first  event  has  occurred. 


P(A  and  B) 


PiA)  P^{B)^--P{B)P,{A). 


(1.8) 


Probabilities  of  two  kinds  enter  into  formula  (1.8):  the  absolute  probability  1 
of  event  A  (or  else  event  B)  coiqputed  regardless  of  idiether  the  event  B  (or  else  evM 

.  -  *  '-^’S 

4)'e--,8^endent'_en  it  has occurred,  and  the  conditional  probability  of  event  5  (or  % 

M 

else  event  1)  computed  on  the  assumption  that  event  A  (or  else  event  B)  occurred.  1 


Therefore  the  absolute  probabilities  P(A)  and  P(B)  are  sometimes  called  the  a  prioril 


(i.e.,  before  the  trial)  probabilities,  and  the  conditional  probabilities  P^  (B) 


and  Pg  (A),  the  a  posteriori  (i.e«,  after  the  trial)  probabilities,  by  the  trial 
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being  meant  here  the  random  experiment  as  a  result  of  which  there  can  be  realized 
the  event  on  which  depends  the  probability  of  occurrence  of  another  event,  dependent 
upon  It. 

The  most  jjqportant  special  case  of  the  rule  of  multiplication  esqpressed  by 
formula  (1.8)  Is  the  case  when  events  A  and  6,  although  compatible,  are  Independent. 

In  such  a  ease  the  occurrence  of  one  of  the  events  In  no  way  alters  the  probability  of 
.  occurrence  of  the  other^iie*,  th&  &  pvixarl  and  a  posteriori  phobe&ilities  become  .equal 
to  each  other: 


I  ) 


P^{B)=P{B),  P^(A)=P{A). 


(1.9) 


For  Independent  events  the  rule  of  multiplication  is  thus  expressed  by  the 
following  formula: 

P(AandB)«=  P(A)-P(B).  (1.10) 


Formula  (1.10)  Is  expanded  for  the  case  of  an  arbitrary  number  of  Independent 


events  B.,  ...B„  : 

12  n 


P(B^  and  B2  and. ..and  B^^)  *=  P(B,)>P(B2). . .  P(B^)  = 


(1.11) 


Let  denote'  an  event  converse  to  the  event  Then  the  occurrence  of  but 

one  (regardless  of  which  one)  of  the  events  ^  excludes  the  possibility  of  the 

simultaneous  occurrence  of  all  the  events  B^,  S^,  Therefore  In  accordance 

with  (1.5)  or  or  ...  or  1 -P(B,2iidB,iand'...andBJ.' 

are  mutually  independent,  then,  accord¬ 


ing  to  the  rule  of  multiplication  •' 

P(B^  and  Ig  •••and  B 


*«t 


Furthermore, 


P(b;j)  *  1  -  P(B^), 
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Consequently. 


P(B^  or  B2  or... or  \)  **  ( 


Fbmula  (1.12)  nakes  it  possible  to  compute  the  probability  of  occurrence  of 
at  least  one  of  the  eoaq>atible.  mutually  independent  events  B^.  62#  •••B^^  according 
to  the  given  probabilities  of  these  events*. 

With  the  aid  of  formula  (1.12)  it  is  possible  very  simply,  for  instance,  to 
f^  the  correct  answer  to  the  question  asked  above  about  the  probability  of  hitting 
the  bullseye  in  a  target  if  both  shooters  fire  simultaneously.  The  sought  after 
probability  is  equal  to 


P(A,  orl,)  .  1  -[1  .  PU,)j  [i-P(A2)] 


Returning  to  the  general  formula  (1.8),  it  will  be  noted  that  it  directly  leads 


P^(B)= 


P(B)Pb(A) 


(1.13) 


Fomula  (1.13)  makes  it  possible  on  the  basis  of  the  absolute  (a  priori)  prob¬ 
abilities  of  two  events  and  relative  (a  posteriori)  probability  of  one  of  them  to 
find  the  relative  probability  of  the  other  event. 

In  certain  problems  it  is  necessary  to  determine  the  probability  of  events 
which  occurs  with  one  of  n  mutually  incompatible  events  B^,  B2.  ...B^  .  These  n 
mutually  ineoiqpatible  events  are  frequently  called  hypotheses  linked  with  the 
occurence  of  event  2.  Using  the  rule  of  addition,  let  us  express  the  probability  of 
event  A  in  the  form  of  tbe  sun 


P(A)=s:P(^endBj,  or  f^endB2, . . .  or  A  and  Bj,)  ^  (AandBy^ ). 


(1.14) 


*  This  formula  generalises  the  addition  rule  for  the  ease  of  eoBq>atible  events.  For 
greater  detail  see  in  /y. 
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Every  tern  of  this  sun  is.  In  accordance  vith  the  nultlplication  rule,  equal  to 

P<A  +  Bit)  *  ^Bit^A) 

and,  consequently  . 

P{A)=^J^P{Bj,)P,^{A).  (1.15) 

Jk»t 

The  relationship  (1.15)  is  called  the  total  probability  fomula.  This  formula 
makes  it  possible  to  deteraine  the  probability  of  event  If  the  a  priori  proba¬ 
bilities  of  hypotheses  ^2*  ***^  ^  posteriori  probabilities  of  event  J, 

an  known,  under  the  condition  that  one  of  the  hypotheses  be  confirmed. 

Let  it  now  be  necessaiy  to  find  the  a  posteriori  probability  of  hypothesis  B^ 
under  the  condition  that  event  ^  has  occurred.  Fbr  this  it  is  sufficient  to  make 
use  of  formula  (1.13) 


P{A) 


Utilizing  also  the  total  probability  formula,  ve  obtain 


P{B,)P„,{A) 


(1.16) 


The  equality  (I.I6)  is  called  the  Bayes  fomula, 

4.  Examples  of  Practical  Applications  of 
Fundamental  Rules. 

As  the  first  important  example  of  the  practical  application  of  the  basic  rules 
of  probability  theoiy  let  us  consider  the  question  of  calculating  the  reliability  of 
operation  of  a  system  of  several  radio-engineering  devices.  The  elements  of  such  a 
system  can  be  tubes,  circuits,  or  stations. 

pacifically  ve  shall  examine  a  radio-relay  communications  line,  consisting  of 
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ii  stations.  A  typical  problem,  nhieh  must  often  be  solved  in  practice,  it  to  find 
the  probability  of  a  breach  in  communication  between  the  terminal  points  of  the 
radio  line  over  a  definite  interval  of  time  T,  with  the  assumption  that  all  stations 
operate  independently  of  one  another. 

Let  the  probability  of  the  breakdown  of  the  k-th  station  (k  =  1,  2, ...n)  over 


the  Interval  of  time  T  equal  Let  us  designate  by  ^  the  event  of  the  breakdown 
within  t^  indicated  interval  of  the  k«>th  station.  Since  the  breakdown  of  at  least 


one  station  breaks  the  communication  between  the  terminal  points  of  the  line,  to 
solve  the  indicated  problem  it  is  necessary  to  find  the  probability 


Rap 


P(B, ,  or  or  ...  or  B^). 
1  &  n 


The  events  ^  are  compatible,  since  during  T  several  stations  can  go  out  of 


operation.  The  use  of  the  addition  rule  for  computing  P  would  not  in  this  case 

“Hap 


be  permissible. 

Let  us  designate  by  ^  an  event  converse  to  Bj^,  i.e.,  that  the  ic»th  station 


operates  without  breakdown  in  the  time  interval  jT*  The  event  converse  to  a  break  in 
communication  on  the  line  consists  in  the  fact  that  all  stations  are  operating  and, 
consequently,  in  accordance  with  (1.5) 


P  *  1  -  P  (1. ,  and  B2  and  ...  and  IL) 
Hap  ** 


It  is  assumed  that  all  stations  operate  independently,  and  therefore,  using  the 
multiplication  rule,  we  obtain 


P(5^  and  and  ...  and  1^)  *  P(B^)  P(B2)...P(Bjj). 

The  probability  of  event  B|c*  as  converse  to  is  equal  to  1  -  j^.  Therefore 

P(B^  and  •••  and  *  (1  -  p,)  (1  -  P2)...(1  -  p„). 


The  sought  probability  of  a  break  in  communication  between  the  terminal  points 
of  the  line  is  equal  to 


1 -*  (I  -  A) (I  - /»a)  • .  (» - 


(1.17) 
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•Toniula  (1.17).  links  the  prol  billty  of  a  break  in  communication  on  the  line  with 
the  probabilities  of  damaf^e  to  its  individual  elements  -  the  intermediate  stations* 
This  result  follows  directly  from  (t.12).  &  the  example  treated,  the  derivation  of 

I 

this  formtila  was  repeated  for  a  concrete  problem. 

If  the  probability  of  the  breakdown  of  one  station  is  low,  and  the  total  number 
of  stations  is  small,  we  obtain  from  (1..17)  the  approximate  equality 

l.e.,  with  low  probabilities  of  damage  to  individual  stations  the  probability  of  a 
break  in  communication  on  the  line  for  any  interval  of  time  7  is  approximately 
equal  to  the  sum  of  the  probabilities  of  the  breakdown  of  the  intermediate  stations 
for  the  same  time  T. 

In  a  special  case,  vten  the  probability  of  the  breakdown  of  any  one  of  sta> 
tions  on  the  line  is  constant,  i.e.,  pj^  a  p  =  const,  formula  (1.17)  takes  the  form 
of 

'  (1.19) 

khen  P«  J  from  (1.19)  we  find 

I  -  ^1  -  I  - e”"'  ss  np.  (1 .20) 

It  can  be  seen  frcn  formula  (1.20)  that,  with  a  large  number  of  intermediate 
stations,  in  order  to  provide  for  sufficiently  hi^  operational  reliability  of  the 
entire  line  it  is  necessary  to  have  rigid  specifications  for  the  reliable  operation 
of  each  of  the  stations.  Thus,  for  instance,  when  the  number  of  stations  on  the 
line  is  n  «  100,  and  the  permissible  probability  of  coBeaunication  interruption  is 
Jkap  *  0.01  the  probability  of  breakdown  of  each  station  must  be  equal  to  p  »  0.0001 . 

It  is  perfectly  evident  that  the  formulas  cited  above  can  be  utilized  for  com¬ 
puting  the  reliability  of  operation  of  a  station  consisting  of  several  units,  or 
for  eoaqputlng  the  reliability  of  operation  of  a  unit,  damage  to  one  element  of  which 
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would  lead  to  Interrupted  operation  of.  the  entire  unit. 

To  raise  the  reliability  of  operation  of  systems  consisting  of  a  large  number 
of  elements,  a  standby  system  can  be  Incorporated  wherein  several  elements  operate 
In  parallel.  It  Is  clear  that  If  the  probability  of  one  element  going  out  of 
operation  Is  equal  to  the  probability  of  the  simultaneous  breakdovm  of  n  lnde> 
pendent  parallel  elements,  idiose  operational  unreliabilities  are  defined  respectively 
by  the  probabilities  «qual  to  the  product  of  pg*  i**-* 

is  considerably  less  thanj^^. 

The  following  exanq>le  Illustrates  the  application  of  the  Bayes  formula  In  prob> 
lems  dealing  with  the  transmission  of  a  telegraphic  signal  with  the  presence  of 
noise  [ 

lAt  us  assume  that  for  the  transmission  of  two  messages  -  '^es*  or  **00"  «  a 
simple  telegraph  system  Is  used  and  that  the  messages  are  transmitted  by  two  dlf* 
ferent  signals,  distinguishable  In  the  receiver  by  the  Hating  of  a  red  and  a  green 
lamp.  If  the  signal  were  always  transmitted  without  distortion  it  would  be  possible 
on  the  basis  of  the  signal  received  (for  Instance  on  the  basis  of  the  lighting  of- — 
the  green  lamp)  to  give  an  unambiguous  answer  to  the  question  of  which  message  was 
sent  (for  Instance.  ’Ves").  l.e..  the  a  posteriori  probability  of  ’Ves"  under  the 
condition  that  the  green  lamp  Is  lit  would  be  equal  to  one. 

Die  to  noise  distortion  the  signal  received  will  not  always  reliably  Indicate 
which  message  was  transmitted,  l.e.,  there  will  be  eases  when  the  green  lamp  lights 
up  at  the  transmission  of  the  message  and  the  red  lamp  at  the  transmission  of 

•yes". 

The  question  Is  asked,  what  Is  the  probability  that  the  operator  decoding  the 
message  (l.e.,  determining  the  message  from  the  signal  received)  does  not  make  a 
mistake  In  asserting,  with  the  lighting  of  the  green  laiq>.  that  the  message  trans¬ 
mitted  was  'yes"!  In  other  words.  It  Is  necessary  to  determine  the  a  posteriori 
probability  of  the  fact  that  the  message  •yes'*  was  transmitted  under  the  condition 
that  the  green  lamp  was  lit. 
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.  ^-J 


(  ) 


A  oonnestlon  with  event  (A)  •  the  lifting  of  the  green  leap  •  tw>  hypotheses 
can  be  expressed:  (B^ )  -  the  message  '^es"  was  transmitted  and  (|^)  -  the  message 
"no**  was  transmitted. 

1b  determine  the  a  posteriori  probablUly  of  the  fact  that  the  message  '^es'* 
was  transmitted  with  the  lighting  of  the  green  laiqp*  it  Is  necessary  to  know  In  ad¬ 
vance  idiat  part  of  the  total  number  of  messages  consists  of  "yes*  messagest  l.e., 
the  a  priori  probability  of  the  transmission  of  the  *^8"  message.  In  addition.  It 
Is  necessary  to  know  the  statistical  properties  of  noise  distortlr^;  the  transmission, 
l.e.,  the  a  posteriori  probabilities  of  the  Hating  of  the  green  lamp  with  the 
transmission  of  the  signal  *yes"  and  of  the  signal  "no". 

Let  us  assume  that  among  the  messages  transmitted,  "yes*  and  "no"  are  encotm- 
tered  In  a  ratio  of  5  :  3*  Then  the  a  priori  probability  of  the  transmission  of  the 
•^es"  message  Is  equal  to 

p("yes")  »=  I 

and  the  a  prion  probability  of  the  converse  event,  l.e.,  the  transmission  of  the 
"no"  message.  Is  equal  to 

P("no")  ■»=  i 

Let  us  farther  assume  that  the  statistical  properties  of  noise  are  such  that 
2/5  of  the  'yes"  meissages  transmitted  are  distorted,  and  1/3  of  the  "no"  messages. 
Then  the  a  postenon  probability  of  the  lighting  of  the  green  laaq)  with  the  trans¬ 
mission  of  the  'yes"  message  (undlstorted  transmission)  Is  equal  to 

(•*««>•)  -  I 

and  the  a  postenon  probability  of  the  lighting  of  the  green  laap  with  the  trans¬ 
mission  of  the  "no"  message  (distorted  transmission)  Is  equal  to 

"  5 

Ihe  desired  a  postenon  probability  of  the  transmission  of  'yes",  under  the 


'5 
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oondition  that  the  green  leap  lights  up,  can  now  be  determined  tgr  means  of  formnla 
(1.13),  lAiidi  foUovs  directly  from  the  moltlplication  rule, 

Cy*-)  -=  K-y**’)  •  fjt,’  Ccr«m«)  (1.21) 

F("green*) 

the  quantities  in  the  numerator  are  determined  bgr  the  statistical  properties  of 
the  massages  and  noise  and  for  the  exaiq>le  under  consideration  are  indicated  abore. 
the  a  priori  probability  of  vhether  the  green  Ijo^  mill  licfht  up  is  not  toomn  in 
advance,  but  it  can  be  co^nited  by  the  formula  of  total  probability*  Actually,  the 
lighting  of  the  green  laiq)  msy  indicate  tmo  inoo^mtible  hypotheses:  either  the 
'^es*  message  mas  transmitted,  or  the  'hio**  Then  according  to  (1*14) 

PC^green")  =*  P('^s*)  ("green")  +  P("no")  Piint>«("green") 

or,  substituting  the  given  values  of  the  probabilities  vULch  characterise  the  mes> 
sages  and  the  noise,  me  obtain  the  a  priori  probability  of  the  lifting  of  the  green 
lamp 

i  *  i  "5  ^  i 

Although  the  *yes*  messages  are  transmitted  more  frequently,  it  turns  out  that 
due  to  the  presence  of  noise  the  lighttng-np  of  the  greea  or  the  red  laiq>  is  eqni- 
probable*  Sow  it  is  not  difficult  according  to  (1*21).  to  find  the  desired  proba¬ 
bility,  i*e,  the  probabUi^  of  the  oorreetaess  of  the  ansmer  mhen  the  green 
lai9  is  lit 

5/g  ?  3/5  3 

'"green"  *  1/2  **  t 

'Ihe  same  result.is  obtained  directly  fbom  (1*l6),  i*e,  from  the  Siyes  fonnila, 
mithottt  the  preliminuy  computation  of  the  a  priori  probability  of  the  lifting  of 
tiie  green  lamp. 

In  the  same  manner,  fomula  (1*l6)  can  be  used  to  find  that  mith  the  lighting 
of  the  red  lamp  the  a  posteriori  probability  that  the  "no*  message  mas  transmitted 
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is  equal  to 


P*»i«ed*’  ("no") 


PC^no")  ("rod") 

P<"no")  P«no»  ("red")  +  P(-yes")  Pj,y^g„  ("red") 

(1.22) 


‘^"rsd" 


("no") 


3/8  •  2/3 


1  1  1 

1?  :  5  -  f 


?«  Series  of  Independent  Tests 


Numerous  problems,  the  solutions  to  vhieh  have  widespread  practical  applica- 


tions,  fit  the  following  system  of  a  series  of  independent  tests.* 


Let  there  be  made  n  independent  tests,  the  probability  of  occurrence  of  random 


event  A  being  the  same  for  each  test  and  equal  to  p.  The  probability  of  the  non¬ 


occurrence  of  event  A  in  the  test,  i.e.»  the  probability  of  the  converse  event,  is 


equal  to  3  «  1-p.  It  is  required  to  find  the  probability  that  after  n  tests  the 
event  A  will  occur  exactly  k  times  (k]^n}.  The  solution  to  this  problem  is  obtained 
by  simple  employment  of  the  addition  and  multiplication  rules. 


Vllth  respect  to  the  order  of  occurrence  of  event  A.  exactly  ^  times,  there  can 


be  expressed  a  number  of  mutually  exclusive  hypotheses.  One  of  these  hypotheses 


(B^ )  consists  in  the  fact  that  event  A  occurs  in  the  first  k  experiments  and  does 
not  take  place  in  the  n  -  k  succeeding  ones.  Then  according  to  formula  (1.11)  we 


,P,(A  and  =  p  9" 

k  times  (n-*)  times 


Axiother  hypothesis  (B2)  consists  in  the  fact  that  event  A  does  not  occur  in  the 


first  test,  then  takes  place  k  times  in  a  row  and  does  not  appear  again  in  the  re¬ 


maining  n  -  k  -  1  experiments.  In  such  a  case 


p^;k  and.B^=q‘P'P...pq'q.. .  q  =  pq” 
k  times  (n-*-*)  5imes 


♦The  term  "test"  is  fully  equivalent  to  the  previously  introduced  term  "random 
experiment"  and  is  employed  here  in  accordance  with  the  tradition  established  in 
the  literature. 
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It.  is  perfectly  obvious  that  for  any  hypothesis  on  the  order  of  occurrence  of 
event  A  exactly  k  tines  and  Its  non«.oceurrence  n  ->  k  tines  In  s  Independent  tests  • 
the  probability  (k  and  §3),  conputed  according  to  fonnula  (1.11),  will  f^ter  the 


appropriate  transposition  of  the  co*nultlples  be  equal  to  p^q''^*^. 


Ihus  any  order  of  the  occurrence  of  event  A.  exactly  Jc  tines  with  jq.  Independent 
tests  turns  out  to  be  equlprobable.  The  nunber  of  possible  hypotheses  on  the  order 
of  the  occurrence  of  event  A  exactly  k  tines  with  n  tests  Is  equal  to  the  nunber  of 
conblnati  ^ns  from  q  elements  to  k.  This  value  Is  usually  denoted  by  the  synbols 
^  or  (  H  }  and  Is  expressed  In  the  fom  of  the  ratio  of  factorials 


Since  the  hypotheses  B2*  ***^(|J)  nutually  exclusive,  by  employing  the 
total  probability  formula  (1.1^)  vc  obtain  the  following  expression  for  the  desired 
probability  Pn(ls)  of  ^e  occurrence  of  event  A  exactly  k  tines  with  n  independent 
experiments 


(i) 


^W-2]^’*(^andB,)=(*)pV-‘. 


(1.23) 


•  It  Is  not  difficult  to  note  that  equal  to  the  coefficient  of  in  the 

expansion  of  the  binomial  (q  +  px)**  by  powers  of  x.  For  this  reason  formula  (1.23) 
Is  frequently  called  the  binomial  formula. 

All  the  possible  quantities  of  the  occurrence  of  event  A  In  s  tests  (k=0,  1, 
...,  n)  constitute  a  set  of  nutually  exclusive  events,  since  In  the  tests  event  A 
either  does  not  occur  at  all,  or  occurs  some  definite  nunber  of  times.  Therefore  in 
accordance  with  (1.4) 


(1.24) 


•>0 
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8'-1 

S 


1 


■4 


■3 


1  -I 


I 


ij 

‘  ^ 


'S 

rd 


Formula  (1*24)  follows  also  from  the  expansion  of  the  binomial  (q  -f  px)"  , 


from  which  when  x  «  1  is  obtained 


The  simplest  example  of  a  system  of  a  series  of  independent  tests  is  the  prob^ 
.  examined  in  Section  4  of  the  reliability  of  a  system  of  eosponents  with  the  eondit 
that  the  reliability  of  any  of  the  components  is  the  same  and  equal  to  3  »  1  -  p.  1 

j 

lach  of  the  Independent  experiments  consists  of  testing,  whether  one  of  the  com-  j 

I 

ponents  of  the  system  has  gone  out  of  commission  (event  A).  The  probability  that 
event  A  will  not  take  place  at  all  (k  «  0)  is,  in  accordance  with  (1.23)  equal  to  | 
(1  *  p)^  •  tdierefrom  the  probability  of  Interrupted  operation  will  be  | 


l-(l-p)". 


which  dess  not  differ  from  (1.19). 


Of  greater  interest  in  practice  is  not  the  probability  of  the  occurrence  of  ah 
event  a  definite  number  of  times,  but  the  probability  that  the  number  of  times  thel 
event  occurs  with  n  independent  experiments  lies  within  the  definite  limits  of  froe 

h  ^  J?2*  .  I 

Bnploying  the  addition  rule,  we  find  this  probability  to  be  i\ 


^(*1  <*<*,)=  PM  +  +  *)+...  +^(*2). 


or,  employing  (1.23) 


(1.25)  j 

A 


6,  Problem  of  the  Humber  of  Channels  in  a  Communication  Line. 

The  following  problem  of  the  necessary  number  of  channels  in  a  conmunlcation 
line  can  serve  as  one  of  the  numerous  examples  of  a  ^stem  of  a  series  of  independ 
ent  experiments. 
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Let  >  eoomunication  line  connect  point  M  with  10  subscribers  at  point  N.  | 

order  to  gain  a  concept  of  the  expected  load  on  this  line,  let  ns  assume  that  each  J 

of  these  subscribers  uses  the  line  for  an  average  of  12  minutes  per  hour.  Then  the  ;| 

a  priori  probability  that  the  line  vill  be  required  is  equal  to  £  »  l/5*  It  can  % 

also  vithin  certain  tolerances  be  considered  that  the  calls  of  any  tvro  subscribers  :| 
are  independent.  Uhat  is  the  probaUllty  that  the  line  will  be  required  simulta.  § 
neously  by  Jjc  subscribers  (k  *  1,  2,  ...  10)?  ;| 

The  solution  of  this  problem  is  provided  by  the  binomial  formula  (1,23).  Here  f 

•#v 

the  event  ^  is  a  call  to  point  from  point  The  series  of  experiments  consists 

Si 

in  testing,  did  or  did  not  each  of  the  10  subscribers  make  a  call.  "'I 

■I 

The  probability  that  the  line  idll  be  required  by  only  one  subscriber  is  equal  Ji 


The  probability  that  the  line  will  be  required  simultaneously  by  two  subscribers'^ 


equal  to 


The  results  of  the  subsequent  calculations  are  presented  in  the  table  immedi¬ 
ately  following. 

Table  1 


^  9 

s 

4 

s 

s 

r 

• 

• 

.-1 

“  1 

0.301 

0.0S8 

0.026 

0,0055 

0,0008 

0,00007 

0,000004 

0,0000001  1 

■  iP 

Having  also  determined  the  probability  that  the  line  vill  not  be  required  by  aty 


of  the  subscribers. 
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it  is  not  difficult  to  see  that  the  sum  of  all  the  computed  probabilities  is^in  ac¬ 
cordance  with  (t, 24), equal  to  unity. 

Above  were  computed  the  probabilities  that  the  line  wauld  be  required  simulta¬ 
neously  by  exactly  Jc  subscribers.  Let  us  now  determine  the  probabilities  that  the 
line  would  be  required  simultaneously  by  not  more  than  {^subscribers  (k=  1,  2,  ... 
10),  for  which  we  utilise  formula  (1.25),  Then  we  obtain 


Pio(0;^A<l)=0,376. 

/»io(0<*<2)=0.678. 

=  0.879. 

‘  />,o(0<A<4)  =  0,967, 

Pio(0<A-i5)==0.993. 

■ /’i(»(0<A»i6)=0,S935. 


Subsequent  computations  will  give  probability  values  differing  from  unity  by 
no  more  than  in  the  fourth  decimal  place.  An  analysis  of  the  values  obtained  shows 
that  it  would  be  pointless  in  the  case  under  consideration  to  have  10  channels  for 
communication'  between  point  ^  and  point  since  the  probability  that  a  connection 
will  be  required  simultaneously  by  more  than  5  subscribers 


/’lo  (6  .=5  A  10)  =  I  -  (0  ^  *  .4  5)  =  I  -  0,993  =  0,007  % 

is  very  small  and  may  in  practice  be  disregarded.  This  result  shows  that  for  practi-  | 
eally  trouble-free  service  to  10  subscribers  (on  the  condition  that  each  of  them  uses(| 

'4\ 

the  line  for  an  average  of  12  minutes  every  hour)  it  is  sufficient  to  have  but  5  1 

I 

channels  on  the  line.  Vflth  such  a  quantity  only  seven  calls  out  of  a  thousand  could 
not  be  immediately  provided  with  a  free  channel,  J 

Figure  1  shows  a  graph  of  P^qC^)  «5  a  function  of  k  when  p  =  l/5»  based  on  the  ^ 

M' 

results  of  the  above  computations.'  This  function  is  determined  only  for  the  integer^ 
values  of  the  argument,  but  in  Figure  1  these  points  are  arbitrarily  joined  ty  a  -| 
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broken  line.  It  can  be  seen  from  this  graph  that  the  function  P^^Ck)  reaches  a 
■axlmum  when  k  s  2,  l.e.t  the  nost  probable  event  Is  that  the  line  be  retired 
siaultaneously  by  two.  subscribers. 


7^ 

I 

1 

t 

r 

V 

^nW _ (e)p*4"  *  — («•- 

.  ,  (b+Dp-* 

-r  A, 


*+l)P_|  L 
jt<f 


(1.26) 


i 

"p 


vi 


••i 

f 

:i 
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Pig.  1.  Probabilities  of  P^o(^)  with^  *  l/5. 

The  presence  of  a  probability  maximuBi.  defined  by  the  binomial  formula  (1.23)» 
is  characteristic  also  of  the  general  ease,  the  value  of  the  most  probable  number, 
jc^.  of  occurrences  of  an  event  depending  both  on  the  number,  n,  of  Independent 
tests,  and  on  the  probability  £  of  the  occurrence  of  the  event  at  each  test. 

To  determine  this  most  probates  number  in  the  general  ease  it  is  sufficient 
to  examine  the  relation 


If  Jt  <  (n  1)£,  then  ^  <k)>  P^  (k  -  1),  and  when  k>  (n  +  Dp.  (k)  <  P^ 

(k-D*  Consequently,  the  nost  probable  number,  l^,  of  occurrences  of  an  event,  at 
which  the  probability  ?||(\0  reaches  a  naxinum,  must  be  equal  to  the  integer  eontainedi 
In  the  value  of  (n  -f  Dg. 
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(1.27) 


Here  the  eynbol  f  jcl  signifies  the  Integer  of  the  number  x.  J 
to  the  example  under  discussion  n  «  10,  £  »  l/5  and  consequently,  =  [y’j”  J 
If  (n  +  1)p  is  an  integer,  then  as  follows  from  (l«25),  there  exist  two  maximuji£| 


probabilities  idiich  are  equal  to  each  other 


Fn(i«b)  *  -  1)« 


^  Ihe 


ototic  Formulas  of  Laplace 


to  eases  where  the  number  of  Independent  tests  is  large,  the  direct  computation^ 

'Tj 

of  probabilities  according  to  fontulas  (1.23)  and  (1.25)  raises  great  difficulties,  | 
since  the  necessary  detemlnatlon  of  the  binomial  coefficients  is  linked  with  the  I 
computation  of  factorials  with  large  arguments.  Thus,  for  instance,  if  in  the  prob»^| 

■t 

lem  examined  above  the  number  of  subscribers  at  point  N  were  not  10  but  100,  for  itsil 
solution  it  would  have  been  necessary  to  compute  the  values  of  (100) t  (99)1  ...  The  | 

value  of  the  factorial  may  be  obtained  with  sufficient  accuracy  by  the  use  of  the  | 

"S 

so«called  asymptotic  foniula  of  Stirling.  | 


This  asysqptotlc  formula  has  the  form* 

•  mf  -w  -e" 


(1.28)  I 


Table  2  presents  a  comparison  of  the  approximate  values  of  factorials  computed! 


by  formula  (1.28)  with  the  exact  values. 


ml  (exact)  ml  by  (1.28) 


Table  2 
Relative  error 

in  ^ 


1 

2 

120 

3.6288- 10» 
9,3326- I0«T 


0.922 
1,919 
118,019 
3.5986- 10* 
9,3249- 10<n 


*  The  oymbolAi  (asymptotic  equality)  signifies  that  the  ratio  of  the  two  expressions! 
related  by  this  symbol  tends  toward  unity  with  the  unlimited  growth  of  m.  I 

.1 
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\h  transfom  the  ULnomial  formula  (1.23)  by  means  of  Stirling*  s  formula 


_ yftM  n*  e~~*  p*  _ 

KSi  **  e“*  J^2ic  (B  —  *)  (n  —  *)"-*  e“  * 


i^erefroffl 


(?)*(,-?-/ 


(1.29) 


Bat  even  vlth  the  use  of  the  approximate  formula  (1.29)  the  computation  of  the 

k  *Jk2 

probabilities  P^dc)  and  particularly  of  sums  like  '  f  Qc)  with  a  large  n 
remains  sufficiently  iabor-c<insuming. 

However,  under  certain  additional  conditions  from  formula  (1.29),  there  can  be 
obtained  a  vexy  simple  asymptotic  formula  idilch  makes  It  possible  to  determine  the 
Indicated  probabilities  with  a  sufficient  degree  of  accura^. 

Let  us  designate  the  deviation  of  the  number  of  occurrences  k  of  an  event  with 
n  Independent  tests  from  Its  most  probable  value  of  k^  by 

^  the  case  of  a  large  n  the  value  of  (n  -f  1)p  differs  relatively  little  from 
its  integer,  and  therefore  idien  n  ^  1  It  Is  possible  to  assume  that  ~  k  •  np. 


(1.30) 


Substituting  (1.30)  Into  (1.29)  and  designating  by  6  the  value 


9=ynpq, 


(1.30*) 


P.TS-9811/V 


22 


we  obtain 


-Ill[/2na*P,(*)K(«p  +  8,-f^)ln(l4-;^)+  ' 

+  («^-«.+|)ln(l-jJ^). 

Since  j  I  ^  3?  I  29  ^  e3q>anding  the  logarithms  into  ex¬ 

ponential  series  and  by  limiting  ourselves  to  the  first  three  terms  of  the  expansion 


we  obtain  after  simple  transformations 


-InlV^P.WI^  1-4(1 -2p)  +  o(l), 


(1.31) 


idiere  Ol  ^  )  is  the  sum  of  all  the  terms  of  the  expansion  in  idiieh  the  order  of 

\<rt}  ^ 

smallness  of  each  item  is  not  lower  than  • 

cr*>  Js  * 

Let  us  now  examine  only  that  range  of  the  values  of  k  for  iMeh  fJL  — >  0 

<r4 

with  an  unlimited  increase  in  n*  Then  from  (1.31)  there  follows  the  asTmptotic 


smallness  of  each  item  is  not  lower  than 


Let  us  now  examine  only  that  range  of  the  values  of  k  for  iMeh 


equality 


i 


(1.32) 


Formula  (1.92),  which  is  sometimes  called  the  Laplace  formula,  is  the  desired 
aty>q>totie  approximation  to  the  probability  ^^(k),  the  exact  value  of  which  is  pro- 
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(  ! 


where  are  assumed  the  designations 


a  — *«~*«  fc — *«— *• 
«  •  . 


Ihe  summation  In  the  right  part  of  (1.3^)  nay  be  regarded  as  an  Integral  one» 
which  ¥iih  a  sufficiently  large  n  will  differ  little  from  the  Integral 

a 

Ihus  the  probability  that  the  number  of  occurrences  of  an  event  with  n  independ¬ 
ent  tests  falls  within  the  limits  of  from  Ic^  to  nay  be  computed  by  means  of  the 
asymptotic  formula 


(1.36) 


idiere  the  values  of  a  and  b  are  defined  in  (1.35). 

Let  us  note  that  with  a  large  number  of  tests  formula  (1.36)  is  applicable  also 
in  those  cases  when  condition  (1.33)  is  violated  for  borderline  values  of  k,  close 
to  and  k2f  if  k^  •  k^  is  large.  This  is  because,  with  a  large  n,  the  probability 
of  the  occurrence  of  an  event  a  precisely  defined  number  of  times  is  extremely  small 
(1.32)3 ,  and  even  large  errors  in  the  determination  of  individual  probabilities 
for  borderline  values  of  k  cannot  substantially  influence  the  overall  value  of 
(k^  ^  k  ^  k2). 

Formula  (1 .36)  is  the  analytic  e]q)reBsion  of  the  so-called  integral  theorem  of 
Laplace.  Assuming  the  designation 

(1.37) 

it  is  possible  to  rewrite  (1 .36)  as 


(®  <  T  ^  ^  (a). 


(1.36) 
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1 
» 

.-s 

!  .'I 

I  I 


'  1 


I 


4 

3 


i 


In  Appendix  II  there  la  presented  a  table  of  the  values  of  F(x}  and  of  the  ln> 


tegrand  function 


The  table  is  prepared  only  for  positive  argunents,  since  the  function  y(x)  is  even» 
and  the  values  of  F(*x}  are  obtained  fron  the  relationship 


(1.39) 


Often  in  place  of  the  Laplace  function  there  is  Investigated  and  tabulated  the 
so-called  integral  of  probability  (function  of  errors.  Kraaq)  function) 


le  function 


t 


=  2f(xJ^2)— 1 


(IM) 


(1.41) 


for  which  the  relationship  (1«39)  is  replaced  by  the  more  simple 

Ihe  function  F(x)  plays  an  extremely  iiqportant  role  in  probability  theozy.  Its 
properties  will  be  presented  in  greater  detail  in  the  foUouing  chapter. 

Fbr  the  solution  of  certain  problems  it  is  convenient  to  make  use  of  the  follow¬ 
ing  special  cast's  of  the  general  fonnila  (1*38) 


(1.«) 


(1A3) 


(1.W) 
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An  inportant  consequence  arises  out  of  formula  (1.44}.  If  oCs  3,  then 


P.'vInr<3)=^(l*-*«U33)=2f(3)-I=0.S 


(1.<»5) 


An  event  ehose  probability  of  occurrence  is  equal  to  0.997  is  customarily  con¬ 
sidered  as  being  practically  certain.  Thus  it  is  practically  certain  that  :fith  a 
large  number  of  independent  eiqwriments  the  deviations  of  the  number  of  occurrences 
of  an  event  from  the  most  probable  mill  lie  vithin  the  limits  of  {ic  -  Icq)  ^  3<r  « 

As  an  illustration  of  the  application  or  the  Laplace  asrBqttotie  fonntlas  (1.32) 
•nd  (1.38)  let  us  return  to  the  probloi  discussed  in  the  previous  section  for  the 
ease  when  the  ntaber  of  subscribers  at  point  H  grows  from  ten  to  a  thousand.  Ike 
solution  of  the  probLem  in  this  case  by  means  of  the  binomial  formula  (1.23)  is  ex¬ 
tremely  cumbersome,  whereas  the  Laplace  asy>q>totic  fomnilas  make  it  possible  to  per¬ 
form  the  necessary  calculations  very  simply. 

Let  eadi  of  the  thousand  subscribers  engage  a  line  an  average  of  six  minutes 
per  hour.  The  a  priori  probability  that  the  line  will  be  required  equals  p  »  I/IO. 
The  most  probable  number  of  calls  equals  in  this  ease 

u  122® 

however,  the  probability  that  exactly  100  calls  will  take  place  is  in  this  ease 
extremely  small.  Actually,  it  follows  from  (1.32*)  that 


YW  V2impq  KCaTlMTol 


.^0,04. 


If  the  line  has  100  channels,  the  probability  that  all  ehaimcls  will  be  occupied  and 
that  at  least  one  of  the  subscribers  will  get  a  "busy**  signal  is,  in  accordance  with 


(1.43)  equal  to 


(-r  >0)  *=  *  -  /'(0)=0,5. 


It  can  be  considered  practically  certain  that  the  number  of  simultaneous  calls 
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will  not  exceed  the  value  of* 

+  3j=  lOa  -f  3  /1UU-U.9=  128.5. 


Vda  resdit  shovs  that  for  the  trouhLe-free  eervieing  of  1,000  sohserlbers,  under 
the  condition  that  eadi  of  then  talks  on  the  average  of  six  ninntes  eadi  hour,  the 
eonunnlcation  Une  should  have  onlj  130  channels. 

It  is  possible  to  solve  the  converse  prohlent  to  how  naiqr  subscribers  can  the 
line  provide  praetleally  trouble-free  service,  if  uith  the  sane  load  (i.e.,  uhen 
p  «  1/10)  the  mmber  of  dunnels  is  increased  to  2C02  Ihe  desired  nonber  of  sub* 
scribers  is  detemined  fron  the  eqaatlM 

200=0, 


uhieh,  oonsldering  (1.27)  and  (1.>>*}.  is  reduced  to  the  quadratic  equation  with  re¬ 
spect  to  <r 


10 


203=0, 


dherefron 


I2ji.ii= ^=144.  ?J=  1 600. 
W  * 


It  is  also  not  difficult  to  provide  an  ansver  to  another  question  of  practical 
interest:  uill  the  total  nonber  of  channels  decrease  if  the  comnieatioo  line  be¬ 
tween  points  K  and  I  were  split  up  into  several  lines,  cadi  of  then  serving  its  re- 
^ective  part  of  the  subscribers  of  point  1.  For  instance,  if  the  line  exanined 
above  were  ^lit  Into  two,  eadi  serving  SOO  subscribers,  the  necessary  nonber  of 
diannels  for  practically  trouble-free  service  on  eadi  line  will  anount  to 

k^Si  +  Zi'Sni^:si70. 

Ihe  total  nokber  of  diannels  in  the  two  lines  will  be  equal  to  IhO.  i.e.,  to 
10  efaannels  nore  than  with  one  eoHsaiication  line. 


*  the  exact  value  of  the  probability  of  the  inequality  cited  belcw  is  0.996. 
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8«  Aaomptotlc  Foraul*  of  Poisson 

m  aumy  prActleal  problems,  pertaining  to  a  system  of  a  large  number  of  independ* 
ent  tests  (n^  1).  the  probability  of  oeeuri'enee  of  an  event  vith  a  single  test  is 
relatively  low*  so  that 

(1.46) 

mhere  is  a  constant  positive  value. 

.  Ih  this  case  the  asymptotic  approximation  (1.32)  for  the  probability 
yields  considerable  errors*  and  it  is  necessary  to  use  a  different  asymptotic  formula, 
arrived  at  by  Poisson. 

Let  us  investigate  the  probability  that  with  n  tests  the  event  will  not  occur 
at  all.  On  the  basis  of  (1.23)  and  (1.46)  this  probability  may  be  presented  in  the 
following  form: 


I 


m 


'■'i 


wherefrom 


p.(0)=(i-p)'=(i_iy, 


(1.47) 


If 


or 


(1.48) 


then  in  the  expension  of  (1.4?),  it  is  possible  to  limit  one's  self  to  the  first 
term;  then 


inalogously 
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U 


'■  ^(i)=<>p(i  -p)— =1-^(1  _p,.^ 


'-r 


I?.(0)^le->. 


and  In  the  general  ease 


*  '  “I  — p  {l—py  *s= 

^«L(«-0...(«t-*-j-n  ^  X* 


Ihe  asynptotle  formula 


^«(A) 


X*  -X 

¥« 


(1.^9) 


I 

.1 

■1 


Is  called  .the  Poisson  formula* 

Pormula  (1.20)  obtained  in  Section  I;  follows  directly  from  (l.U?)  when  k  =  0,  if  | 
it  is  assumed  that  the  number  n  of  stations  is  large,  and  the  probability  p  of  the 
going  out  of  service  of  one  station  is  small. 

A  table  of  values  of  the  Poisson  approximation  (1.49),  for  the  binomial  formula 
with  several  values  of  A  and  k,  is  presented  in  Appendix  I. 

Kaking  use  of  (1.49),  let  us  vrite  an  ejqpression  for  the  probability  of  the 
occurrence  of  an  event  not  more  than  m  times: 


*-«  *.o 


Ihe  last  sun  may  be  presented  as  an  integral 


.  *-o  •  r 


(1.50) 
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vVtor^4»(Wri«V  . 


I-I^n  I  -■'-f  rr*-rii«  iffii 


1  I 


n 


suggesting  sn  Integral  presentation  of  the  gamma-function  J"*  (m  ■(•  1 ) ,  tMch  for  in¬ 
tegral  values  of  the  argument  coincides  vith  the  factorial 


r  (m  4, 1)  =  m! = J  2*"  e"'</2. 


(1.51) 


When  Xs  0,  the  integral  in  (1.50)  does  not  differ  from  (1.51)* 
The  integral 


^  M 

r(/n4-|.l)=J  2'"e'"</2=»r(m4-I)_J  z”t~*dz 


(1.52) 


u  » _ I  r('»*  +  U^ 

^  j)  * 


(1.53) 


using  the  Stirling  formula  for  ^  kl,  ve  obtain 


ln/»^(A)^*lnI~l-lln2)r-^*-f  ljln/i4-A  = 


Designating  k  -X  =  ^  1^  »  let  us  rewrite  the  last  equality  in  the  fora  of 
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■•■rM 


is  called  the  incomplete  gamma-function.  Dy  means  of  this  function  the  probability 
of  the  occurrence  of  an  event  not  more  than  m  times  may  be  presented  in  the  follow¬ 
ing  manner: 


I 


I 

■J 


There  exist  detailed  tables  of  the  Incomplete  gamma-function  for  a  vide 
range  of  variation  of  its  arguments.  A  table  of  values  of  the  probabilities 
p^(k<m),  obtained  with  the  aid  of  said  tables,  is  presented  in  Appendix 

I. 

In  several  applications  of  the  Poisson  formula  the  parameter  X  is  much  greater 
than  unity,  although  it  satisfies  the  relationship  (1.48).  In  such  cases  the 
Poisson  approximation  (1.49)  coincides  with  that  of  Laplace  (1.32). 

Actually,  from  (1.49)  there  follows 

lnP,{A)  — Alnl  —  i  — InA!: 


1 

I 

I 


1 


■■I 


-fir 

'  ^ 
‘5^ 


lnP,(A)  v. (5^4.14.1  +  ln(2»:i). 


.  If  It  Is  now  assumed  that  (l.e.»  those  values  of  k  are  being  examined, 

which  are  relatively  close  to  the  most  probable  value  of  X  ),  then  expanding  ln(l  + 
Sk  il 

^JL  )  In  a  series  of  powers  of  ^  and  limiting  ourselves  to  the  first  tern  of  the 

X  X 

expansion,  we  obtain 


I*  I 

InP,(A)-w-^-^ln(2ni). 


wherefrom 


which  coincides  with  the  Laplace  form  of  approximation  for  the  binomial  formula. 


The  Shot  Effect 


intervals  of  time,  fluctuates  about  a  certain  mean  value.  As  a  result  of  this 


phenomenon,  idtich  is  called  the  shot  effect,  there  take  place  random  voltage  fluc¬ 


tuations  in  the  diode  load.  It  is  asked,  what  is  the  probability  that,  during  a 


certain  interval  of  time  t,  exactly  k  Aectrons  reach  the  anode?  The  answer  to  this 


question  is  provided  by  the  Poisson  formula. 


Let  tis  assume  that  the  intensity  of  the  electron  stream,  i.e.,  the  average  num- 


The  number  of  electrons,  emitted  by  the  cathode  and  reaching  the  anode  in  equal 


ber  V  of  electrons  reaching  the  anode  per  unit  of  time,  is  known.  Over  a  sufficient¬ 


ly  large  Internal  of  time  T,  the  number  of  emitted  electrons  will  equal  n  *  V  T.  | 
These  a  electrons  emerge  in  the  indicated  time  interval  ^  at  random  and  independently  < 
of  each  other.  Khat  Interests  us  is  the  probability  that  k  of  p  electrons  will  e-  | 

merge  in  a  fixed  interval  t,  iddch  constitutes  a  part  of  | 

J 

The  emergence  of  n  electrons  may  be  regarded  as  n  consecutive  tests.  The  event  I 
whidx  either  occurs  or  does  not  occur  as  a  result  of  each  test,  is  that  a  given  1 
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(  I 


^,.t^^&'j*g.’'f,p^lMjSSjw^^ji^vjiikygd^l^^iji;iAk4?igj^’.«^’^gs!CT-iy.yi^^  -< ,  "j-fn 


■1 


•leetron  appears  in  the  tiss  interval  t»  The  probability  that  any  of  the  electrons 


appears  in  the  time  interval  t«  i«e.,  the  probability  of  the  occurrence  of  event  A 


vlth  each  test,  is  equal  to  p  »  ^  ^ 


i 

J 


Thus  ve  return  again  to  the  system  of  a  succession  of  independent  tests.  A  pec£ 


liarlty  of  the  problem  in  question  is  the  large  number  of  independent  tests  n  and  thf' 


loir  probability  of  the  occurrence  of  an  event  with  a  single  test.  Then,  according  t^j 


the  Poisson  formula  (l.lt9),  substituting  X. »  *  ift,  we  find  the  probability  that» 

(1.5U) 


during  the  time  interval  ^  exactly  k  electrons  reach  the  anode 

There  is  another  vay  of  solving  the  problem  at  hand.  Preserving  the  condition 


of  the  Independence  of  electron  departure,  let  us  subdivide  the  time  interval  t 

t 


into  N  sections,  with  a  duration  of  ^  t^-  each,  and  assume  that  the  probability 
of  departure  of  one  electron  during  the  time  ^  1,  equals  yet.  The  probability  of 
departure  of  two  or  more  electrons  is  an  extremely  small,  secopd-order  one  with  re¬ 
spect  to  At.  The  probability  that  an  electron  will  not  reach  the  anode  in  the  first 

^  'I 

interval  A  t  is  equal  to  l-v^At,  and  the  probability  P(0)  that  an  electron  will  not 
-  *  N  I 

reach  the  arnde  in  any  one  of  intervals  by  virtiie  of  its  independence  is  equal  to  | 

the  product  of  JJ.  co-imiltdpliers  of  (1  -T"  At),  i.e.. 


(0)  =  (I  -  Vi/)*  =  (1  _  Vi/)"  . 

Letting  now  N — r  os>  or  i  t^— ^  0,  we  obtain 

P(0)  =  e-''. 

i.e.,  formula  (1.5^)  with  k  s  0.  In  an  analogous  manner  it  is  possible  to  obtain 
expressions  for  P(1),  £(2),  and  in  the  general  ease  for  P(k),  which  leads  to  the 
Poisson  formula. 

The  problem  of  the  number  of  electrons  reaching  the  anode  is  one  of  a  broad 
class  of  problems,  in  which  there  is  determined  the  probability  of  the  occurrence 
of  an  event  a  definite  number  of  times  during  a  certain  time  interval  1.  To  this 
class  belong  problems  conceming  the  number  of  telephone  calls  made  over  various 
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Internals  of  time,  or  concerning  the  nvunber  of  disintegrated  radioactive  atoos 


The  Poisson  fonaula  is  also  used  for  solving  the  problem  of  the  number  of  sign 
changes  in  the  period  i  in  a  generalized  telegraph  signal  (Fig.  2).  vhich  is  charac¬ 
terised  b|7  the  fact  that  the  current  i(t)  may  take  only  tvo  values,  -ffa  and  -h.  but 
that  the  instants  of  sign  change  are  random.*  If  V  is  the  average  number  of  sign 
changes  in  a  unit  of  time  and  if  the  probability  of  a  sign  change  in  an  interval  1 
(equal  to  does  not  depend  on  the  pattern  of  sign  changes  outside  this  interval, 

for  the  computation  of  the  probability  that  during  the  time  t  there  will  occur  ex¬ 
actly  k  sign  changes,  by  a  reasoning  analogous  to  that  carried  out  above,  we  ob¬ 
tain  formula  (1.5^). 


Fig.  2.  Generalised  telegraph  signal 

10.  Generalisation  of  the  System  of  Independent  Tests.  The  Concept  of  the 

Karkov  Chain 

There  has  up  to  now  been  examined  a  succession  of  independent  events  vhlch  vas 
characterised  by  the  occurrence  or  non-occurrence  of  a  definite  event  A  with  each 
test.  Jsi  certain  cases  the  outcome  of  each  test  can  be  not  one  of  the  two  events  A 
or  I.  but  a  set  of  a  incompatible  events  A^.  us  assume  that  the  prob- 

ability  of  occurrence  of  event  ^  with  each  test  is  to  jjj^(i  *  1.2....n). 

Since  the  events  A^.  Az****^  comprise  a  set.  x  *  1* 

*  Sometimes  instead  of  the  number  of  sign  changes  there  is  examined  the  number  of 
seros.  i.e..  the  intersection  of  the  i{%)  curve  with  the  abscissa.  It  is  clear  that 
the  two  types  of  examination  are  equivalent  to  each  other. 
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Vhat  is  the  probability  that»  as  a  result  of  n  Independent  experiments,  event 


'a 


'H 


^  occurs  exactly  tines,  event  ^  appears  exactly  times,  etc.,  with  event  ^ 


■k 


appearing  exactly  tines?  The  answer  to  this  question  Is  provided  by  the  formula 


i 


P'‘  P*’  •  •  • 


(1.55) 


idiich  is  a  generalization  of  the  binomial  formula  (1.23).  The  numbetsk^,  k2».*.l^ 
must,  naturally,  conform  to  the  condition  of 

(1.56) 


-ll 


.  *i4-*2-r  ••  •  +  *«=«• 

When  m  -  2  formula  (1.55)  turns  into  (1.23).  ^ 

An  example  of  a  generalized  qrstem  of  independent  tests  can  be  found  in  the  sue|>| 

s 

eessive  transmission  of  the  letters  of  the  Russian  alphabet  with  the  condition  that  || 
each  succeeding  letter  is  transmitted  independently  of  its  predecessor.  The  result  | 

I 

of  a  test  is  the  appearance  of  one  of  33  letters  which,  in  the  example  at  hand,  eon«  | 


stltute  a  set.  The  question  can  be  asked,  what  is  the  probability  that  In  a  text  ofl 


3 


B  letters  the  letter  A  occurs  k^  times,  the  letter  B  occurs  k^  times,  etc.,  with  the| 
letter  la  occurring  k^^  tines.  The  answer  to  this  question  can  be  obtained  with  the:| 
aid  of  formula  (1.55).  if  there  is  available  a  table  of  probabilities  (relative  fre-;| 

-I 

quencies)  of  the  occurrence  of  the  letters  in  the  Russian  language.  A  table  of  {| 


this  sort  was  compiled  by  P.  P.  Frakhov,  according  to  idwse  data,  for  Instance,  the 


I 


probability  of  occurrence  of  the  letter  A  is  equal  to  2^  »  0.075,  the  letter  B  ^2 

I 

0*017,  etc. 

The  example  at  hand  also  well  illustrates  the  possibilities  of  the  further 

I 

generalization  of  a  system  of  independent  tests.  The  necessity  of  such  a  general-  i 
ization  is  linked  with  the  fact  that  in  any  language  the  sequence  of  letters  is  not| 

4 

independent.  | 

the  simplest  case  It  can  be  assumed  that  the  occurrence  of  one  of  the  33  I 


letters  of  the  alphabet  in  a  given  test  depends  only  on  what  letter  occurred  in  the: 


preceding  test,  and  does  not  depend  on  what  letters  had  occurred  in  earlier  tests,  | 
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A  tuecessicn  of  tests  like  that,  for  irtdch  the  relative  probability,  in  the  (S  +  Oth 
test,  of  the  occurrence  of  one  event  of  a  set  of  events  depends  only  on  uhat  event 
had  occurred  in  the  S-th  test,  and  does  not  change  as  a  result  of  additional  lnfor« 
nation  as  to  uhat  events  had  occurred  in  earlier  tests,  is  called  a  simple  Markov 
eku^.  For  homogenous  Harkov  chains  the  relative  probability  of  a  transitton 
fro*  event  ^  to  event  ^  depends  only  on  these  events,  but  does  not  depend  on  the 
number  of  the  experiment.  If  the  ftissian  language  is  to  be  regarded  as  a  siiq[>le 
homogenous  Harkov  chain,  its  statistical  structure  may  be  described  as  a  collection 
of  33  e  priori  probabilities  Pj^  of  the  occurrence  of  the  l-th  letter  of  the  alpha¬ 
bet  and  of  33^  transitions  P^^^,  i.e.,  a  posteriori  probabiliUes  that  the  i-th  letter 
of  tlie  alphabet  mill  be  foUoved  by  the  ^J-th  letter. 

The  next  Increase  in  conq;>lexity  consists  in  the  consideration  of  the  probabili¬ 
ties  of  combinations  of  three  letters,  tut  not  of  more  than  three.  The  occurrence 
of  a  letter  with  a  given  test  depends  only  on  what  letters  were  transmitted  in  the 
preceding  two  tests,  but  does  not  depend  on  what  had  occurred  earlier.  Here  the 
relaUve  probabiUty  must  be  i.e.,  the  a  posteriori  probability  that  after  the 
two-letter  combination  of  the  -i-th  and  j-th  letters  of  the  alphabet  there  will  follow 
the  l£-th  letter  of  the  alphabet.  Continuing  in  such  a  manner,  it  is  possible  to  ob¬ 
tain  ever  increasingly  complex  Harkov  chains. 
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Chapter  II 


UUS  OF  PROBABIUTr  DISTRIBUTION  OF  RANDOM  VARIABLES 
1«  Definitions  and  Temlnoloey 

The  subjects  of  investigation  In  the  preceding  chapter  vere  random  events#  idd.eh 
either  occurred  or  did  not  occur  as  the  result  of  a  random  experiment.  Di  practice 
it  is  frequently  more  convenient  to  designate  the  result  of  a  random  experiment  fay 
a  certain  real  variable  ^  ,  iMch  is  called  a  random  variable.  Thus,  for  instance, 
a  result  of  the  observation  of  noise  in  radio  receivers  in  a  fixed  sv^ment  of  time  is 
characterised  fay  a  random  momentary  noise  voltage.  Another  example  of  a  randou 
variable  is  the  number  of  occurrences  of  an  event  with  jj.  Independent  tests.  As  in 
the  ease  of  a  single  random  esqperiment,  it  is  impossible  to  predict  the  exact  value 
of  a  random  variable.  It  is  possible  only  to  predict  the  statistical  rules  of  the 
change  of  a  random  variable,  i.e.,  to  assign  probabilities  to  its  values. 

The  range  of  possible  values  of  a  random  variable  may  be  finite  or  infinite,  but 
countable  (i.e.,  these  values  can  be  determined  by  means  of  a  natural  series  of  num> 
bers).  Then  the  random  variable  is  called  discrete.  The  possible  values  of  a 
random  variable  may  occupy  an  entire  continuous  range  (finite  or  infinite),  so  that 
it  is  iiqpossible  to  number  these  values.  Then  the  random  variable  is  called  contliva* 
ous. 

Let  the  number  of  events,  occurring  in  result  of  a  random  experiment,  be  finite. 
In  accordance  with  this  ire  will  assign  a  certain  single  number  to  each  event. 

Then  all  the  possible  values  of  a  certain  discrete  random  variable  will  correspond 
to  a  set.  The  probability  that  a  discrete  random  variable  will  take  one  of  the  pos¬ 
sible  values  will  be  singly  equal  to  the  probability  of  occurrence  of  a  random  event 
corresponding  to  that  value. 

If  the  range  of  possible  values  of  a  continuous  random  variable  is  broken  down 
into  a  finite  number  of  segments,  the  aggregate  of  events  wherein  this  variable  falls 
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In  toy  of  these  segments  constitutes  n  set.  Then  the  introduction  of  the  concept  of 
probability  that  the  values  of  a  random  variable  lie  vithin  the  limits  of  a  certain 
segment,  is  completely  analogotts  to  the  discrete  ease.  It  obviously  makes  no  sense 
to  speak  of  the  probability  that  a  continuous  random  variable  vUl  take  a  given 
value,  since  this  probability  equals  zero,  speaking  of  probability  in  connection 
vith  a  continuous  random  variable,  it  is  always  neeessazy  to  bear  in  mind  a  certain 
segment  of  the  complete  range  of  its  possible  values. 

^  order  to  have  a  sufficiently  complete  understanding  of  the  statistical  rules 
characteristic  of  the  random  experiment  at  hand,  it  is  thus  necessaiy.  first,  to 
know  the  range  of  possible  values  of  the  random  variable  characterieing  a  random  ex¬ 
periment.  and  second,  the  probabilities  of  these  values. 

The  lav  that  for  each  possible  value  of  a  discrete  r:.idoK  variable  or  for  some 
range  of  values  of  a  continuous  raxvdom  variable,  there  is  a  corresponding  probability 
that  the  random  variable  will  take  these  values  (or  else  will  lie  in  some  range  of 
possible  values),  is  called  the  law  of  the  probability  distribution  of  a  random  vari¬ 
able. 

The  analytical  expressions  of  distribution  laws  are  distribution  functions, 
which  for  discrete  random  variables  are  functions  of  an  integer  argument,  and  for 
continuous  random  quantities  are  functions  of  a  contlnv  >us  argument. 

Let  us  examine,  for  instance,  the  number  of  k  occurrences  of  a  random  event 
with  Q  independent  tests.  This  number  is  a  discrete  random  variable,  the  possible 
values  of  which  are  equal  toksO.V**B*  As  has  been  sliovr.  in  the  preceding  chap¬ 
ter,  the  probability  f^Ck)  of  the  occurrence  of  an  event  times  with  n  Independent 
tests  equals  ,■ 

M*)=C)pV-. 

This  foniula  establishes  the  lav  that  for  each  possible  value  of  k  there  is  a  cor¬ 
responding  probability  P|^(k),  i.e..  the  lav  of  the  probability  distribution  of  the 
number  of  occurrences  of  an  event  with  n  independent  tests.  This  law  is  frequently 
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eall«d  the  hinonlel  lav  of  probabdUity  diatribation 


2«  Distribution  ghnetlons  of  Random  Variables 

Let  tts  assune  that  the  randoa  variable  £  Mgr  take  values  vithin  a  range  of 
tton  •  ea  to  -f  eo  and  let  us  exanine  sone  levd  2,  idilch  can  vary  vitbih  the  saae 
Halts.  Ihe  present  assunqptlon  does  not  restrict  the  generalllgr  of  the  results  given 
belov,  since  the  variation  of  a  randoa  variable  within  a  Halted  Interval  of  values 
will  siaq;>l7  slgnliy  that  the  probability  of  Its  reaching  any  cone  of  the  nuaerical 
axis  outside  the  Indicated  interval  Is  equal  to  zero. 

The  fhnetlon 


F{x)^P{lKx), 


(2.1) 


idiich  shows  bow  the  probabill^  that  the  values  of  a  randoa  variable  do  cot  exceed 
the  selected  level  of  x  depends  on  the  aagnltude  of  that  level*  Is  called  the  to- 
ftmctlon  of  probabllltv  distribution 


Thus*  for  Instance*  for  a  discrete  randoa  variable  vhich  can  with  equal  prob¬ 
ability  take  any  of  the  values  of  a  natural  series  of  nuabers  f  row  0  to  the  in¬ 
tegral  function  of  distribution  vill.  have  the  fora  (Fig.  3) 

0.  x<0 

..  F{x)=P{l<x)^  x=-.0,  I.  2.  ...  « 

/ 

I, 


9  t  X  y  4  S  X 

Fig.  3*  Iav  of  unlfom  distribution  of  a  discrete  randoa  variable. 
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If  the  Integral  distribution  ftenetion  for  a  continuous  randoa  variable  baa  the 


fom  (Fig.  4) 


F(x)=PCi<x) 


0.  x<0 

0<x-h 

t.  *>A. 


it  is  said  that  this  continuous  randon  variable  is  evehlx  distributed  over  the  in* 
terval  (0*  h).  Bcanples  of  integral  functions  of  distribution  for  a  discrete  randon 
variable  can  also  be  found  in  fonmlas  (1.42)  and  (1.33)  of  the  preceding  duster, 
bgr  neans  of  uhich  there  is  dstendned  the  irobabililar  that,  with  a  large  modier  n  of 
independent  eaqperinents,  the  nunber  of  ocenrrenees  of  an  event  does  not  exceed  ■  4  fi. 
Ustributions  of  the  discrete  randon  variables  given  bgr  fomnlas  (1.32)  and  (1.49) 
are  called  respectively  the  laplace  and  Poisson  distributions. 


i 


c  • 


lig.  4.  lav  of  unifom  distribution  of  a  continuous  variable. 

Let  us  point  out  the  basic  properties  of  integral  distribution  fbnetions.  Ihe 
values  of  these  fbnetions,  vhidi  represent  probabilities,  aust  lie  vithin  the  Units 
of  trou  0  to  1,  vbere 


f(-eo)=P(t^-^)=0. 

is  the  probabiUty  of  an  impossible  event,  and 

<oo)=  I. 


(2.2) 


(2.3) 
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is  tlie  probstdlitj  of  a  certain  air«ot. 

fho  property  oaprossed  ty  oqaality  (2.3)  is  an^ogous  to  the  property  of  a  set 
of  events* 

If  the  randon  TariabiLe  lies  belov  the  level  of  Z2>S|»  there  exist  t:«>  nat^ially 
exelasive  possihillties:  XLther  this  randon  variable  is  located  b^v  ?evel  or 
it  is  located  betueen  levels  and  Then,  asine  the  z«le  of  additioc,  ne  obtain 
P(5<s*)=P(5<*,  ^ 


eherefroe 


(2A) 


or,  in  view  of  (2*1). 


P(xt  <i<x^^P(x^  -  F  {xt). 


(2.5) 


Ibtts.  the  probability  that  a  randon  variable  is  confined  vitliin  definite  Units 
is  equal  to  the  difference  of  thr^  values  of  the  integral  distribation  function  for 
the  vpper  and  lower  Units.  Foraala  (1*3B)  in  the  preceding  diapter  is  a  special 
case  of  (2*5).  when  the  Laplace  function  serves  as  the  integral  distribation  function. 


Since  the  left  side  of  the  eqpiaUtyr  (2*5)  nqr  not  be  negative, 

F(aj)>f(x,)  when  X2>x,. 


(2.6) 


eonseqioentlj*  the  integral  function  of  distribation  is  always  a  non-dininishing 
function. 

Ihe  integral  distribation  function  of  the  discrete  randon  variable  (  is  dis« 
continuous,  it  grows  in  steps  with  those  values  of  2  wtiich  are  possible  values  of  C  • 
ne  integral  distribation  ducted  in  fig.  3  ia  in  this  sense  typical  of  the  discrete 
randon  variable. 

ne  integral  distribation  function  of  a  contimoas  randon  variable  is  oontinoous 


and  differentiable,  ne  derivative 


(2.7) 
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l8  called  the  probabiUt/  density  or  the  differential  distribution  function  of  a 
continuous  random  variable*.  The  probability  density,  as  the  derivative  of  a  non- 
diminishing  function  (the  integral  distribution  function)  cannot  take  negative  values. 


i.e. , 


m{x)>0. 


(2.8) 


The  term  "probability  density"  becomes  understandable  by  examining  the  change 
in  a  continuous  random  variable  vlthin  sufficiently  narrow  limits  from  z  to  x 
Then 

•  W—  -JJT - Si 


or,  considering  (2.5),  we  find 


(2.9) 


from  expression  (2.9)  vith  a  small  dz,  it  can  be  seen  that  u)  (s)  fulfills  the  mean¬ 
ing  of  probability  density,  since  it  is  equal  to  the  ratio  of  the  probability  of  a 
random  variable  falling  withi.n  the  interval  d  2,  to  the  length  of  that  interval. 


Fig.  5*  Distribution  curve  of  a  continuous  random 
variable.  Shaded  area  is  equal  to  the  probability 
that  values  of  the  continuous  random  variable  lie 
within  the  intervaKz^,  Zg)* 

•  From  here  on  oi  (x)  is  called  simply  the  distribution  function. 
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Let  us  represent  the  distribution  function  (probability  density)  in  the  form  of 
a  distribution  curve  (Fig*  5)*  Integrating  both  parts  of  (2*7)  vithin  the  limits  of 
from  *«otox,  and  bearing  in  mind  (2*2)*  we  obtain 


S 

P(5<s)«f(x)=  Jw(x)dx 


(2.10) 


and*  bearing  in  mind  (2*5)*  ve  obtain 


P(xt<i<x^—^w(x)dx. 


(2.11) 


Thus  the  probability  that  the  values  of  a  continuous  random  variable  lie  vithin  a 
given  interval  is  equal  to  the  area  under  the  distribution  curve  of  that  random  vari¬ 
able  for  the  interval  in  question  (ef.  shaded  part  of  Fig.  5).  In  the  special  ease 
vlth  s  -  eo  and  x^  *  oo»  it  follovs  from  (2*11)  that 


m 

J«(X)(fx=s 


(2.12) 


i.e.t  the  total  area  under  the  distribution  curve  is  always  equal  to  unity.  In  order 
to  ensure  the  convergence  of  the  iiiq>roper  integral  (2.12).  it  is  necessary  that 


Urn  w(x)s=llina;{x)=0, 

r-»-«  Mtmt 


(2.13) 


i.e*,  it  is  neeessaz7  that,  in  those  cases  when  the  random  variable  can  vazy  vithin 
infinite  limits,  the  abscissa  in  both  directions  be  an  asymptote  of  the  distribution 
curve, 

A  distribution  curve  6)  (s)  may  have  one  or  more  maxima.  A  maximum  of  <i}  (x) 
corresponds  to  a  range  of  the  most  probable  values  of  a  random  variable.  The  value 
of  C  =  X  ^  ,  for  idiich  the  probability  density  reaches  a  maximum,  is  called  a  node. 

It  is  simplest  of  all  to  check  the  above-enumerated  properties  of  distribution 
functions  (integral  and  probability  density)  on'  the  example  of  uniform  distribution 
(Fig.  4),  The  probability  density  in  this  ease  is,  in  accordance  with  (2.7)  equal  to 


F-TS-9811/V 


x<0,  x>A, 
0<x<h, 


l.e.,  Kainttlns  a  constMt  valua  In  the  interval  (0,  ^),  thus  Justifying  the  appela< 
tion  of  unlfora  distribution. 

One  remark  should  be  aade  eonoeming  the  designations  employed.  Random  vari¬ 
ables  and  the  arguments  of  their  functions  are  frequently  designated  by  the  same 
qrabols*  iddeh  can  lead  to  misunderstandings.  Therefore  from  here  on  random  vari¬ 
ables  vUl  be  designated  by  Greek  letters ^  ....  and  the  arguments  of  their 
distribution  fimetion  by  Latin  letters*  Most  frequently  encountered  in 

application  is  the  so-called  normal  law  of  distribution  of  a  random  variable,  to 
vhich  the  following  section  is  devoted. 


1.  Homal  Law  of  Distribution 


The  probability  density  of  a  continuous  random  variable  which  has  a  normal  law  | 

of  distribution  depends  on  two  parameters  £.  and  <r  .  and  is  determined  by  the  formula  1 

•| 

.  ft’ 

u— )»  I 

»(x)=sce  .  I 


‘4 

The  constant  c  cannot  arbitrary,  but  must  be  determined  in  such  a  manner  as  1 
to  satisfy  condition  (2.12).  l.e..  | 

i 


m  U-M)' 

•**  dx  -. 
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Since 


«  U^)»  «0  _£* 

J«  ~^dx^9  Je  'dz=9yru*. 


it  follows  that 


and,  oonsequently. 


(2.14) 


ftnetlon  (2.14)  with  £.»  0  was  used  above  with  the  asymptotic  fomula  of  Laplace 
{[ef.  formula  (1.32)^.  A  table  of  the  values  of  this  function  with  a  3;  0  and  <r  ^  1  is 
presented  in  Appendix  n. 

Pig.  6  shows  the  probability  distribution  curve  for  the  nomal  law  together  with 
some  parameters  characteristic  of  it.**  The  curve  has  one  maximum  at  the  point  x  »  », 


idterebgr 


wla)=w  = — \=- , 

w  w  _  , 


(2.15) 


At  a  distance  ot  J  from  the  mode  2  »  a  the  probability  density  Is  only  0«004,  and 
the  area  under  the  curve  o)  (x)  in  the  band  of  +  Jar  comprises  99.7^  of  the  total 
area  under  the  distribution  curve,  i.e.,  it  may  be  considered  with  a  probability  of 
0.997  that  any  random  variable,  distributed  according  to  the  normal  law,  lies  between 
1*3^  nnd  i  >  3  o~  •  (3h  the  asymptotic  approximation  of  Laplace  this  property  was 


denoted  by  fomula  1.45). 


*  Actually 


-J.jrr’dr-lbc. 


••  -£!±2l 


>■10  I* 


rdr  i%  m 


**  The  curve  j  *  called  the  Gaussian  curve.  The  normal  law  of  distribution 

is  also  often  called  the  Gaussian  law. 
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Fig.  6.  Probability  density  curve  for  normal  distribution 
(figures  between  arrows  indicate  the  percentage  of  the  total 
area  under  the  distribution  curve  comprised  by  the  area  of 
the  corresponding  strip). 

It  is  not  difficult  to  show  that  the  points  of  inflection,  at  idiich  the  dlstri» 
button  curve  has  a  maximum  slope,  are  determined  from  the  equality 


(2.16) 


It  can  be  seen  from  (2.15)  and  (2.1 6)  that  the  larger  the  parameter  .  the 
smaller  is  the  maximum  probability  density  and  the  further  is  the  point  of  Inflection 
from  the  mode,  i.e.,  the  more  widely  dispersed  are  the  significant  values  of  function 
cj  (x).  Conversely,  the  smaller  the  parameter  o’  ,  the  higher  the  maximum  probability 
density,  and  the  closer  is  the  inflection  point  to  the  mode,  i.e.,  the  more  compact 
are  the  significant  values  of  the  function  to  (^ ,  or  the  greater  is  the  probability 
that  a  random  variable,  distributed  according  to  the  normal  law,  falls  on  a  strip  of 
a  given  width  which  includes  the  point  x  »  a.  Fig.  7  presents  for  comparison  the 
probability  density  curves  of  the  normal  law  of  distribution  for  three  values  of  c"  • 
The  envelope  of  this  set  of  curves  is  the  pair  of  hyperbolas 


Y2h9  ‘»  —  a* 
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vhose  a^ptotes  are  the  straight  line  x  »  a  and  the  abscissa.'  Wiw  oo  the  distri¬ 
bution  curve  merges  with  the  abscissa*  and  when  0  it  turns  into  the  delta-func¬ 


tion* 


nmw(x)s=  lim-T^e  **’  =*(x— a). 


(2.17) 


-s  -♦  -z  0 


Pig.  7.  Conqparison  of  normally  distributed  probability 
density  cur/es  with  various  parameters  <r  . 


The  Integral  distribution  function,  l.e. ,  the  probability  that  a  random  variable; 
distributed  according  to  the  normal  law  lies  below  the  level  of  x,  is  in  accordance 


with  (2.10)  equal  to 


k  K  - 


(2.18) 


The  function  (2.18)  with  a  «  0  and  <7'»  1  was  examined  in  the  preceding  chapter 
in  connection  with  the  asymtpotic  approximation  of  Laplace  (1.37)  for  the  binomial 


*  Jr  view  of  the  fact  that  the  delta-function  will  in  the  future  receive  widespread  | 
use,  some  of  the  principal  properties  of  this  function  are  presented  in  i^pendix  IV.  | 
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[541? '  'JftRTlSj^f '  '1''' '" 


JB^I"  ■'sW'-  '■vj^ 


lav  of  distribution*  (A  table  of  the  values  of  F(x)  is  shovn  in  Appendix  II.) 

MLth  the  above>nentioned  a^yi^totie  approximation  in  mind,  the  normal  law  of 
distribution  magr  be  regarded  as  a  limiting  lav,  toward  >dtich  tend-  the  probability 
distributions  of  the  numbers  of  occurrences  of  events,  when  the  total  number  of  in¬ 
dependent  tests  grows  without  limit* 

Pig*  8  shows  a  graph  of  the  integral  distribution  function  P(x),  which  cor¬ 
responds  to  function  (2*18)  with  0  and  o'  =:  1.  This  function  grows  monotonously 
from  0  to  1,  idiere,  at  the  origin  of  the  coordinates  . 


f(0)=4.f(oo)=4.. 


The  sum  of  the  values  of  this  function  at  two  points,  located  ay^^trically  with  re¬ 
spect  to  the  origin  of  the  coordinates  (Figure  8)  is  equal  to  unity,  which  corre¬ 
sponds  to  the  previously  presented  formula  (1*39)*  Close  to  the  origin,  tho  integral 
distribution  f\mction  F(x)  has  a  sector  which  comes  close  to  being  linear j  it  is  well 
described  by  several  initial  terms  of  the  exponential  series 


(2*19) 


With  a  sufficiently  large  argument  there  takes  place  the  a^mptotic  e;q>ansion 

#1  t  1C  \ 

(2*20) 


*•/  »  1  •  .  *  /  •  >  L  3  >5  ,  \ 


Let  us  note  that  if,  in  an  alternating-signs  series,  we  are  to  restrict  ourselves  to 
several  terms,  the  error  will  be  less  than  the  value  of  the  first  rejected  term.  It 
follows  from  (2.20)  that 


•-rfc*  <^W<' 


/2 


'.XX 


Pig.  9  there  are  presented  for  comparison  curves  of  the  integral  distribu- 

t 

tion  function  for  the  same  values  of  O'  as  in  Fig.  ?.  Whenc/— >0^  the  integral  dis¬ 
tribution  curves  asymptotically  approach  the  abscissa  with  x  <  a,  and  a  parallel 
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fig,  8,  Ourve  of  the  integral  normal  law  of  distribution. 


f-O 

4 


Pig.  9.  Comparison  of  integral  normal  distribution  curves  with 

various  parameters  r  • 

straight  line,  F  ?  1 ,  when  x  a,  Ihe  limiting  curve  has  the  form  of  a  tmit  Jump 
at  the  point  x  =  a.  This  signifies  that  no  matter  how  small  is  the  magnitude  of 
>  0,  the  probability  that  the  random  variable  a  -  £  ,  is  equal  to  zero,  and 
it  is  certain  that  C  ^  i  *  S  •  Consequently,  this  random  variable  is  the  constant 

1* 

The  corresponding  probability  density  of  the  constant  ^  is  equal  to 

w(x)  =  i{x-~a). 

It  should  be  noted  that  the  delta-function  satisfies  all  the  requirements  laid  down 
for  distribution  functions,  in  particular  (cf.  i4>pendix  IV) 
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Ji(jt--a)rfx=l. 


Thus,  constants  nay  be  regarded  as  random  variables*  whose  distribution  func¬ 
tions  are  delta-functions. 

4,  Joint  Probability  Distribution  of  Random  Variables 
to  now  we  have  discussed  the  probability  distribution  of  only  one  random 
variable  ^  .  One  may  in  an  analogous  manner  examine  the  Joint  probability  distri¬ 
bution  of  two  dependent  random  variables 


P(;<x,  n<_y)  =  f2(x,  y). 


(2.21) 


idilch  yields  the  probability  of  combining  the  following  two  events:  the  random  vari¬ 
able  (  Is  located  below  the  level  of  x  and  the  random  variable  7^  is  located  below 
the  level  of  The  function  FgCxtX)  is  called  the  two-dimensional  integral  prob¬ 
ability  distribution  function. 

If  in  (2.21)  it  is  assumed  that  ^  s  oo*  then  F^^.oo)  will  be  equal  to  the  prob¬ 
ability  that  ^  is  located  below  the  level  of  x  with  any  position  of  ^  *  i.e. .  coin¬ 
cides  with  the  integral  distribution  function  of  the  random  variable  ^ 


inalogously* 


fa(x,  oo)=f„(x)*, 

y)=FM- 


(2.22) 


(2.22* ) 


from.  (2.21)  it  follows  also  that 


F^i(— oo.  y)=^2(x.  -oo)  =  0 


(2.23) 


*  such  eases  where  a  one-dimensional  distribution  function  has  a  double  index, 
the  first  figure  indicates  the  order  of  the  function,  and.  the  second  figure^  the 
number  of  the  random  variable  whose  distribution  is  determined  by  the  function  at 
hand. 
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and 


F,(<»,  oo)==l. 


(2.24) 


It  is  possible  to  resort  to  a  geometrical  interpreta-vion,  regarding  the  variablea  4 
and  as  coordinates  of  a  point  on  a  plane  (Fig.  10).  Then  the  tvo-dimenslonal  in¬ 
tegral  distribution  function  F2(x.2)  yields  the  value  of  the  probability  that  a  point 
with  the  random  coordinates  (^,7^)  vill  lie  in  that  part  of  the  plane  which  is 
shaded  in  Fig.  10.  The  probability  that  this  point  will  lie  within  a  rectangle  with 
vertices  at  the  points  (x^,  j^),  (xg.  J^).  (xg,  yg),  (x^,  jg),  will  equal 


P(x,<5<Xa.  y,<T,<y,)= 

=  yi)  + 

Fiixx.yi). 


(2.25) 


Formula  (2.25)  generalizes  formula  (2.5)  for  a  one-dimensional  case;  when  s 
-  oo*  22  (2.25)  turns  into  (2.5). 


Fig.  10.  Pair  of  random  variables  as  coordinates  of  a  point  on  a  plane. 

For  continuous  random  variables,  the  two-dimensional  integral  function  XgCi.Z^ 
is  continuous.  The  mixed  second-order  derivative 


v.fx  v>— 


(2.26) 


is  called  the  two-dimensional  probability  density  or  the  two-dimensional  distributlohl 


function  for  the  random  variables  §  and  7^ 
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The  two'dlmensional  distribution  function  is*  Just  like  the  one-dimensional*  al' 


vagrs  non-negative 


w,(x.  y)5.0. 


(2.27) 


The  tNO-dimensional  distribution  function  is  expressed  in  terms  of  the  prob- 
abilitjr  density  by  means  of  the  double  integral 


^a(x,  y)—  J  ^tV2(x,  y)dxdy. 


(2.28) 


— a>-.o 


Let  us  represent  the  two-dimensional  distribution  function  as  a  sur¬ 

face  (Fig.  11).  Then  the  probability  £(A)  that  point  A*  with  random  coordinates 
whose  probability  density  equals  ^{^(x.j).  falls  within  a  region  G  of  the  plane,  is 
equal  to  ^e  volume  bounded  by  the  sector  of  surface  a^^Cx.y)*  idtose  projection  on 
the  plane  coincides  with  area  G,  i.e.. 


y)dxdy. 


(2.29) 


The  total  volume  under  the  distribution  surface  is.  in  accordance  with  (2.24)  always 
equal  to  unity 


m  m 

J  y)dxdy=\. 


(2.30) 


11 

-rU\ 


■m 


f 


I 


{  ! 
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fig.  11.  Probability  distribution  surface  of  a  continuous  random 

variable. 
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]bi  order  to  ensure  the  convergence  of  the  improper  Integral  of  (2.30),  it  is 
necessary  that  the  surface  asyvqptotically  approach  the  plane  of  (x,;)  in  all 

directions. 

Qjr  neans  of  a  given  two-dimensional  density  difficult  to 

find  the  probability  distribution  laws  of  each  of  the  random  variables.  In  fact, 
from  (2.22)  and  (2.28)  we  find  for  the  random  variable^ 


idiere 


fu(x)—  j  jwt(x,  y)dydx,==:  Cw,i(x)dx, 

-  «o 

«» 

»ii(x)=  jo'aix,  y)dy. 


(2.31) 


(2.32) 


inalogously  for  the  random  variable 

^i»(y)=  I  Jwa(x.  y)dxdy=  fw„(y)dy, 

n  im  v 


idiere 


(2.33) 


(2.3^) 


Let  us  present  separately  the  fomulas,  important  to  practical  applications,  of 
the  joint  distribution  functions  of  two  independent  random  variables. 


Ihe  random  variables  ^  and  ^  are  called  independent,  if 
f,(s,  y)  =  p(5<x,  •n<y)=/^i,(x)f„(y). 


(2.35) 


where  f{^(x)  and  integral  distribution  functions  of  each  of  the  random 

variables  under  examination.  An  analogous  formula  holds  true  also  for  the  two-di¬ 


mensional  probability  density 


•«(«.  y)=wu(x)  w„(y). 


(2.36) 
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where  <0  2(x)  and  ^^2^^  dlstrihition  functions  of  each  of  the  randon  Tariables 
under  examination. 

Thust  two  randon  wariables  are  independent,  if  their  joint  distribution  function 
is  the  product  of  the  indiwidual  distribution  functions. 

Ihe  simplest  example  of  a  two>dimensional  distribution  function 


I 


=  1  (*— «)W— cl*  c<y<d. 


i^(x.  >)  =  {  (*— «)(rf— c) 

0  .  x<o,  x>b,  y<r.  y>d 


corresponds  to  the  uniform  distribution  of  two  random  wariables  in  the  rectan^e 

The  probability  that  point _A  with  the  coordinates  ^  and  falls  into 

a  region  lying  within  rectangle  G,  lAll  be  equal  in  this  ease  simply  to  tiie  ratio 

of  the  areas  of  region  g  and  rectangle  G. 

The  studbr  of  the  joint  distribution  laws  of  random  rariables  may  be  generalized 

for  XL  randon  rariables  •••  f  ^  .  la  this  ease  the  Q^dinensional  integral 

distribution  function  Fq(x^iX2,...^}  yields  the  probability  that  the  random  rariable 

,  X|  and  also  that  the  random  rariable  ^  2  below  the  lerd  of  X2>  and  the 

random  raidable  H-ca  below  the  level  of  3^  etc,,  i,e,, 

\  P(?i<Xi,  ^<x„  ...  x„  ...  xj.i 

*  *  '  p 

The  randon  variables  ^  ^  ^  2.*  called  mutually  inu?pcndent  if 

F,(x,.  X,.  ...  xJ  =  f„(x,)f„(x,)...fj,(xJ. 


(2.37) 


where  ia  the  integral  distribution  function  of  the  randon  variable  ^  s 


1 ,2,,,,^), 


Ibmula  (2.38)  generalizes  the  pr«./ious^  presented  forK>2la  (2.35). 

If  m  of  IL  arguments  of  function  (2.37)  turn  to  infinity,  i.e.,  if  it  is  known 
that  i  of  n  random  values  nay  be  arbitraiy,  then  this  function  becomes  the  integral 
distribution  function  of  the  remaining  n  •  1  random  variables 

f^(oo,...  x^^,,  ...xj.  (2.39) 
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u 


(  K 


from  the  definiUon  (2,3?)  it  foUovs  that 


and 


— ao.  •••  x^=.. 

=  -Jff.  ...--«))=0 


(2.h0) 


f,(oo.  oo,  ...oo)=l. 


(2.h0») 


If  tha  n-dinensional  Integral  fonetion  has  the  derlTattve 


r  tr  r  ^  t _  •••X*) 

r.(x,.  ... 


than  this  dariratlYe  is  called  the  n-dlnensional  densitgr  distribution  or  the  £>di- 
■enslonal  distribution  function  of  the  randoe  variables  ^  ,  t  ,,,,  B ^ . 

The  integral  distribution  function  (2.37)  is  expressed  in  tems  of  the  prob- 
abilitgr  density  bj  Means  of  tr  n*aultiple  integral 

^0(^1*  *  *  *  ■*•») 

(2>2) 


«•  *0 


~  1  J"*  J *'*^*‘‘  *** 


Ihe  aggregate  of  the  randon  values^^  ^  Mgr  be  treated  as  the  oo» 

ordinates  of  a  point  or  as  the  co^wnents  of  aTeetorin  n-dinensional  space.  The 
probability  f  (4)  that  point  A  vith  randon  coordinates,  whose  probability  density  is 
equal  to  lo^(g|,X2***»XQ)>  falls  in  the  r*igion  ^ot  the  space,  equals 

.J-.  (X|,  x,.  ...  x^dxydx^  - .  -dx^. 

Silking  use  of  (2.h0*),  we  obv^ln 

m  m  m 

J  J"  rfi|rf*,...dx,=  |. 


(2.^3) 


(2.li4) 


If  the  raadon  YariabLes  f ,,  are  antualty  independent,  it  follows 

fnm  (2.3B)  that 


(2.45) 


p.fs-9eii/v 


( 


nhers  probability  density  of  the  randon  value  ^  *  1»2#...b) 

It  follows  from  fonmlas  (2.39)  and  (2.42)  that,  with  m  xi. 

(jf  J.  •  •  •  ®®)  *  •*• 

~  J  J  ‘  *  •  J  (•*^1*  6*X|  dxj  .  . .  dx^, 


idiere 


-C«  — «0  — • 


“’ll,  (^h  •*2*  ♦  •  •  X„)  — 

«e  « 

~  J***  *  *  *  ■*ii)dX|„ j., . .  .dx^. 


(2.46) 


(2.47) 


n—m  pa3 

Thus,  knowing  the  n-dinensional  distribution  function,  it  is  always  possible  to 
obtain  the  distribution  function  of  any  group  of  is.  random  variables  (1^  a  ^  a)  by 
means  of  integrating  within  the  infinite  limits  the  n«dimensional  function  with  re» 
spect  to  the  remaining  n-m  variables. 


5.  Maltidimensional  Normal  Law  of  lAstribation 


The  n«dimensional  normal  law  of  distribution  provides  an  example  of  the  joint 

probability  distribution  of  a  system  of  random  variables  §  ,  ^ which  is 

/A  yi 

of  the  gi'^stest  importance  in  practical  application.  The  ji-dlmensional  distribution 
function  corresponding  to  the  normal  law  is  by  definition  equal  to 


®«  Xjt . .  .x_)—  • 


I 


■•arS 


xt-ai 


l-t  *-l 


*1*1  •••*<, 


(2.48) 


where  D  is  a  determinant  of  the  ii«th  order  which  has  the  following  form 

0  = 


1  »’u.  .  .r,^ 

'’n  J  •  •  •'’a, 


'’<11  '■«*•••  J 


M' 


(2.49) 


and  the  quantity  is  the  co-factor  of  the  element  r^  in  the  determinant  H. 

Thus  the  n^dlmensional  distribution  function  (2.48)  depends  on  the  2  p  para¬ 
meters  and  on  the  parameters  i.e.,  on  a  total  of 

parametei’s. 
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If  the  random  variables  f  ^  f*  ?7i  independent  and  each  of  them  is 
distributed  according  to  the  normal  law  with  the  respective  parameters  ijj,  1, 

2,...n),  It  foUo»s  fron  (2.14)  tai  (2.45)  that  the  Joint  n-dlmenslonel  distrllMtlon 
of  these  random  variables  is  determined  ly  the  function 


■ 


K2k«i* 


»,(*!.  Jf,. . . .  Jfj  = - X - 


I  ^  <**“•*)’ 

-t2j  .» 

*-i  * 


(2.50) 


Formula  (2.50)  is  a  special  case  of  the  general  formula  (2,48)  with  r^y^  *  0(3t  4 
for  which  C  =  1 ,  D^k  =  0  when  1  f  k,  and  =  1  with  i.  *  Is.  When  n  *  1  formula 
(2*50)  turns  into  (2.14). 

Let  us  now  examine  in  greater  detail  the  normal  law  of  distribution  of  two  ran¬ 
dom  variables.  A  two-dimensional  distribution  function  depends  in  this  case  on  5 
parameters:  a^,  a^,  £^3  *  -*  determinant  D  is  equal  to 

and  the  co-factors  ®11  *  ^22  *  -12  *  ?21  “ 

Ft’om  the  general  formula  (2.48)  ^  obtain,  when  n  *  2,  the  following  expression 

for  the  two-dimensional  distribution  function  of  two  normally  distributed  random  . 


values 


Vi  — f* 


..  »*(*!♦  Jf»)  = 

’  C  .  .  m 


(2.51) 


the  :  rface  of  a  two-dimensional  normal  distribution  which  corresponds  to  (2.51)  has 
the  form  depicted  in  Fig.  12.  At  the  point  x^  *  ?i>.  ?2  *  #2  toe"probability  density 
is  !atw  its  maxlnum  «id  is  equal  to 
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w,(a,,  aj  Wj^  . 


(2.52) 


Prom  this  point  the  distribution  surface  approaches  the  plane  (x^,  Xg)  monotonously 
from  all  directions.  Any  vertical  intersection  of  the  surface  i  *2^ 

a  plane  passing  through  the  point  of  maximum  probability  density  represents  a  Gaus¬ 
sian  curve  (cf.  note  on  p.  h6  ).  The  probability  density  maintains  constant  values, 
along  the  ellipses  iMch  are  horiiontal  sections  of  surface  (2,51).  Ibe  equation  of 
a  family  of  ellipses  of  equal  probability  densities  has  the  form  of 

^ - 2''  Sjj;  +  -*•  (2.53) 

Various  but  constant,  for  a  given  A  ,  values  of  the  probability  density  correspond 
to  various  values  of  the  parameter  A>  0  for  all  points  of  ellipse  (2.53).  When 
A  >  0  the  ellipse  degenerates  into  the  point  x^  =  a^ ,  x^  »  of  maximum  density , 
and  as  A  Increases  the  intersecting  horizontal  plane  is  lowered  ever  more,  tlie 
density  on  the  ellipses  of  equal  probabilities  undergoing  a  corresponding  decrease. 
The  lengths  of  the  large  and  small  half-axes  of  an  ellipse  of  equal  probabilities  are 
proportional  to  the  parameter  A  ,  and  are  equal  to 


a— 

P  —  iTt— 


rr=-r  *“»'r+7- 


(2.54) 


Fig,  12.  Surface  of  two-dimensional  normal  law  of  distribution. 
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Jk' 


-»f 


The  probaMlily  that  a  point  of  the  plane  (x^t  *2^  with  random  coordinates, 
distributed  according  to  the  two*dinensional  normal  law,  will  lie  within  an  ellipse 
of  equal  probabilities  with  a  fixed  parameter  X  ,  is  equal,  according  to  (2.43), 
to 

X  ti. -r,) 

(2.55) 


KM 


irtiere  g  (X)  is  the  area  of  the  plane  (x^,  x^)  bounded  bjr  the  ellipse  (2.53)  • 

The  Integral  (2.55)  is  computed  by  resorting  to  polar  coordinates.  As  a  result 
of  such  a  computation  we  obtain 

X* 

P{i)=  I  _  e“*“A 

Esqxressing  the  parameter  X  in  terms  of  the  lengths  of  the  half •axes  of  the  equal- 
probability  ellipse,  we  find  also 

•) 


(2.56) 


Completing  the  square  in  the  expression  in  brackets,  we  obtain 


U,-a,)» 


^rS.,rT=7fJ  ”'’1  2(1-'*) 


idierefrom 


I  ^ 


(2.57) 


P.TS-9611/V 


60 


It  is  not  difficult  to  show  that  each  of  the  mutually  dependent  random  variables, 
distributed  according  to  the  two-dimensional  normal  law,  is  also  distributed  normally. 
Substituting  (2.51)  into  (2.32),  we  find 

»„  ,x,)=  J  WAX,.  x,)dx,  =  e"  X 


•A. 


I 


I  ^ 


11 


-5>l 


( . 


i 


o: 


inalogously 


{*»-•>)* 

I 


(2.58) 


If  the  random  variables  are  independent,  then  in  accordance  vith  (2.50) 


_  I  rUi-g.)*  (jr,-<t,)»  I 


(2,59) 


Formula  (2.59)  is  a  special  case  of  (2.51)  when  r  *  0.  Uhen  <r^  -  O'  the 
ellipses  of  equal  probability  turn  into  circles  with  radius  ^  ~ ‘r'. 

The  probability  that  a  point  on  a  plane,  whose  coordinates  are  independent, 
normally  distributed  random  variables,  lies  within  a  circle  of  radius y*,  whose  center 
is  at  point  (a^,  §2),  is  according  to  (2.56) 

(2.60) 


jL 


P(p)  =  l-e  p>0. 

In  this  case  it  is,  as  can  be  seen  from  (2.60),  practically  certain  that  a  point 
with  the  indicated  coordinates  will  fall  in  a  circle  with  a  radius  of  3  tdiose 
center  is  at  the  point  ,  Xg  * 

.  W 


0  1  ,  r  3^ 

Fig.  13.  Rayleigh* s  Integral  Distribution  Function 


the  probability  density,  nddch  corresponds  to  the  integral  distribution  function 
(2.60),  equals 


,{p)='^  =  X. '  (2.6,) 

Distribution  function  (2,61),  iMch  is  often  called  the  Rayleigh  function, 
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-/ 


Ci 


l-.iiasy,  j* 


determines  the  probability  that  two  independent  random  variables,  distributed  in  ac¬ 
cordance  with  the  noraal  law  having  equal  parameters  vy  =  =  <r'*  ^*11  within  a  ring 

bounded  by  the  tws  concentric  circles  of  radii  andj>+4j*,  whose  center  lies  at  the 
point  of  maximum  Joint  probability  density  of  these  random  variables. 

Fig.  13  shows  several  curves  of  the  integral  Rayleigh  distribution  function 

(2.60),  and  Fig.  14  -  the  respective  probability  density  curves  (2.61).  The  maximum 
_ I 

density  a>  *  attained  when  P=  o'*  This  is  the  point  of  inflexion  of  the 


J 


Integral  distribution  curve.  The  slope  of  the  probability  density  curve  at  the  ori¬ 
gin  is  equal  to  ui\o)  -  ^ 


.A  • 


The  envelope  of  a  set  of  Rayleigh  distribution  func- 

X 

tion  curves  is  a  hyperbola,  whose  equation  is  y  =  jlj>  . 

w(Wf 


If  '  ^ 

Fig.  14.  Probability  Density  for  a  Rayluigh  Distribution 

Let  us  present  in  conclusion  the  expression  of  the  Joint  probability  distri¬ 
bution  function  of  three  normally  distributed  random  vazdables.  Prom  the  general 
formula  (2.48)  when  n.  =  3  we  obtain 

Jf2.  ^3)  = 


rr-4-_  • 


(2.62) 


where 


2  (I  —  rjj  —  f|,  —  f|j  +  2r„rar„) 


(x,-a,)*  + 


(*j  -  «*)*  (^3  -  03)*  - 
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sir"'  -»■  "-*4^tt  '-•w^ 


>  t 


(\ 


i(rti~‘ravfn) 


«i»i 


<JCa--aj)(x3-.a3)- 


Vi 


(«3-03)(JCi-ai)]. 


(2.63) 


k  three-dlfflensional  distribution  function  »  x^.  x^)  depends  on  as  naror  as  9 

paraaeters:  ij,  §2.  *3.  » ^^2*  t\z*  ^zy  -31* 


6.  Mtoerical  Characteristics  of  a  Random  Variable. 


The  integral  distribution  function  (or  its  derivative,  the  probability  density), 
examined  in  the  preceding  sections,  provides  a  coiiq[>lete  characterization  of  a  random 
variable,  as  it  points  out  the  range  in  idiich  the  values  of  the  random  variable 
change,  and  the  probability  of  finding  it  in  specific  sections  of  that  range.  How. 
ever,  in  a  number  of  cases  it  is  sufficient  to  know  much  less  about  a  random  vari¬ 
able,  an  overall  concept  being  adequate. 

This  is  analogous  to  the  case  idien  Instead  of  a  description  of  the  most  minute 
details  of  a  solid,  ono  deals  only  vith  such  of  its  overall  numerical  characteristics 
as  length,  width,  height,  volume,  moment  of  Inertia,  etc*. 

In  the  probability  theory,  certain  numerical  constants,  obtained  according  to 
specific  rules  from  the  distribution  functions,  serve  as  overall  characteristics  of 
a  random  variable. 

As  such  numerical  characteristics  of  random  variables  it  is  customaxy  to  con¬ 
sider  the  so-called  distribution  moments  of  various  orders.  For  continuous  random 
variables  the  distribution  moments  of  the  Jc-th  order  are  determined  according  to  the 
formula* 

.  J  x*a)(x)dx 


(2.64) 


under  the  assumption  that  the  i]iq>roper  integral  converges.  The  numbers,  m^,  may  be 
treated  geometrically  as  moments  of  inertia  of  the  corresponding  orders  of  a  plane 
figure,  bounded  by  the  abselsia  and  the  curve  2  «  ^  (x). 


*  The  symbol  mj^(  ^  )  denotes  not  a  function  of  the  random  variable  ^  ,  but  yi  averag¬ 
ing  operation  of  the  variable  E  from  the  totality  of  its  possible  values  Iconpare 
p.  104  ,  formula  3.48). 
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If  the  random  variable  is  discrete  and  takes  the  values  of  vith 

the  probabilities  of  £2***«JE*n*  moment  of  distribution  is  equal  to 


(2.65) 


The  siiiq>lest  numerical  characteristic  of  a  random  variable  •  a  distribution 
moment  of  the  first  order,  deteralning  the  abscissa  of  the  gravitational  center  of 
a  distribution  curve,  is  called  the  mathematical  expectation  or  the  mean  value  of  a 
random  variable.  In  accordance  vith  the  general  definition,  the  mean  value  of  a 


continuous  random  variable  is  equal  to 


and  the  mean  value  of  a  discrete  random  variable  is  equal  to 


(2.66) 


(2.67) 


The  difference 


between  a  random  variable  and  its  mean  value  is  called  the  deviation  of  a  random 
variable.  The  deviation  of  a  random  variable  is  equal  to  the  distance  from  the  ab> 
scissa  of  the  gravitational  center  of  the  distribution  curve  of  this  random  variable. 
Distribution  moments  of  the  deviation  probabilities  of  a  random  variable  are  called 
central  moments  and  are  designated  by  j  .  In  accordance  with  (2.64). 

m 

~  J  W rf*-  (2.68) 

In  distinction  from  the  moments  ^  || j  of  a  distribution  curve  vith  respect  to  the 
coordinate  axis,  Wiich  are  called  initial  awments  (or,  simply,  moments  ftr^  ),  cen¬ 
tral  moments  are  the  moments  of  the  distribution  curve  with  respect  to  the  axis 
passing  through  the  gravitational  center  of  this  curve. 

For  a  discrete  random  variable  the  central  moments  of  distribution  are  defined 
respectively  by  the  summation 
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(2.69) 


«.M1=2('.-'»|>V.. 

If  the  mean  ralue  of  a  randoa  variable  is  equal  to  eero«  then  ^  and  the 
central  nonents  of  distribution  coincide  vith  the  noments.  It  is  obvious  that  a 
first  central  nonent  is  alvays  equal  to  zero. 

accordance  vith  the  general  definition,  a  first  nonent  for  a  continuous  ran¬ 
dom  variable  is  equal  to 

«• 

J**®^'^*  (2.70) 

and,  for  a  discrete  random  variable,  to 

(2.71) 

r-l 

k  second  central  moment  of  distribution  is  called  the  dispersion  of  the  random 
variable^  ,  and  is  defined  in  accordance  vith  (2«68)  and  (2.69)  fay  the  formulas 

J  (x~m,)*a;(x)dx  (2.72) 

for  a  continuous  random  variable  and 

Af,{t|=V(x,-.mi)*p^  (2.73) 

r?l 

\ 

for  a  discrete  random  variable. 

Ihe  magnitude  V  is  called  the  mean  Quadratic  or  standard  deviation  of  the 
random  variable  ^  from  its  mean  value.  In  physical  applications  it  is  customary  to 
call  the  square  root  of  the  dispersion,  the  fluctuation  of  a  random  variable.  The 
ratio  of  the  deviation  of  a  random  variable  to  the  standard  deviation  Is  called  the 
normalized  deviation  of  a  random  variable. 

The  central  second  moment  and  the  initial  second  moment  are  not  independent. 
They  are  linked  a  relationship  idiich  follows  directly  from  (2.'/2)  ^and,  analo¬ 
gously  for  a  discrete  random  variable,  from  (2.73)}  : 
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«0 

Af,=  J  (**->2xm,+mJ)tt;(x)rfjc  = 

«-«0 

^  **  IQ 

■^x^w{x)dx~~2m^  ^xw{x)dx+m]  ]w{x)dx. 


and  since 


^  Oi 

J xw(x)dx=m,,  J w {x) dx  —  I, 


M^—tn^ — wj, 


(2.74) 


Ihus  the  dispersion  of  a  random  variable  is  equal  to  the  difference  between  its 


second  moment  of  distribution  and  the  square  of  its  mean  value. 


In  the  following  sections  there  are  examined  properties  of  the  mean  value  and 


of  dispersion,  and  these  numerical  dxaracteristics  are  defined  for  the  distrihition 


functions  examined  above. 


^  Examples  of  Oomputation  of  Mean  Value  and  Dispersion  of  a  Random  Variable. 


Let  us  find  the  mean  value  and  the  dispersion  of  a  discrete  random  variable. 


distributed  according  to  the  binomial  law  (1.23).  Making  use  of  formula  (2.6?),  we 


find* 


and  since 


As  it  represents  the  sum  of  the  probabilities  of  a  set  of  events,  the  latter  sum¬ 


mation  is  equal  to  1  end,  consequently. 


OT,=:n/>. 


(2.75: 


*lhe  summation  starts  at  k  s  1,  since,  when  »  0,  the  corresponding  term  in  the 
summation  is  equal  to  zero. 
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Comparing  (2»75)  with  (1.2?),  we  conclude  that  the  mean  value  of  the  numbers  of 
occurrences  of  an  event,  with  n  independent  tests,  differs  from  tlie  most  probable 
number  of  occurrences  by  less  than  one. 

The  dispersion  of  a  discrete,  binomially  distributed  random  variable  is  formed 
according  to  formula  (2.74).  For  this  we  first  determine  the  initial  second  moment 

"»= r »’( 2  )/’*«'■*= S  *  (;z  v-' = 

The  first  summation  equals  (n^>l}£,  as  the  average  number  of  occurrences  of  an  event 
with  (n  •  1)  tests,  and  the  second  equals  the  sum  of  the  probabilties  of  a  set  of 


events,  lierefore 


Bien  the  value  of  the  dispersion  ^  of  the  occurrence  number  of  events  with  n  in¬ 
dependent  tests  is  equal  to 

Mt=np  (rt  —  l)p  4-np -'{op)-  =  «P  — — P) 


AU—npq. 


(2.76) 


5 

Coi^paring  (2.76)  with  (1.30*)  we  find  that  the  parameter  <r  in  the  Laplace  asymp¬ 
totic.  formula  is  the  dispersion  of  the  number  of  occurrences  of  an  event  with  n 
independent  tests. 

The  mean  value  of  a  continuous  random  variable ,  distributed  according  to  the 
normal  lav  (2.14),  is  in  accordance  with  (2.66)  equal  to 

“'“Taj''' 
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The  integral 


(  \ 


V 

f 


ae  ^'du  —  Q, 


since  the  integrand  function  is  odd»  and  the  integral 


Ka 


L-  f 

2c«  J 


40  M* 


e  du  —  I 


hgr  virtue  of  (2.12).  Therefore  the  nean  value  of  a  normally  distributed  random  vari¬ 
able  is  equal  to 


m,=a. 


(2.77) 


Thus  the  mean  value  of  a  random  variable,  distributed  according  to  the  normal  lav,  is 
equal  to  the  parameter  a  of  that  lav. 

The  dispersion  of  a  normally  distributed,  continuous  random  variable  is,  in  ac> 
cordance  vith  (2.72),  equal  to 

(*-a)» 

Af.  ■  " 


f  jo* 


OB  a* 

iJt' 


and,  after  Integrating  by  parts,  ve  obtain 

Af,  =  9*. 


(2.78) 


Thus  the  significance  of  both  paramete  ^  the  normal  distribution  function  has 
been  determined;  one  of  them  is  the  mean  value  of  a  randbm  variable,  and  the  other  is 
its  dispersion. 

The  mean  value  and  dispersion  of  a  continuous,  Rayleigh-distribution  (2.61)  ran* 


dom  variable  can  be  found  vith  equal  ease 

s* 


m|  =  ^Jx»e  **  dx— 9 
Af,=-^  j  — e  *‘^dx  = 


(2.79) 
(2.79* ) 


I 

.'3 


^41 

'A 


I 

I 


I 

I 

I 

* 


I 


8.  Coefficients  of 


end  Eiccess 


Krom  the  definitions  provided  in  Section  6  and  also  ftom  the  exaiqples  cited  in  4 
Section  7  it  foUoirs  that  the  mean  value  of  a  random  variable  provides  a  conception  of  | 
the  domain  of  its  most  prohab3.e  values,  and  the  dispersion  points  out  the  extent  to  i 
ivliich  the  values  of  the  random  variable  are  scattered  with  respect  to  its  oean  value.  | 
3a  other  words,  the  mean  value  and  the  dispersion  are  the  characteristics  of  location  ^ 
and  scattering  of  the  lav  of  distribution  of  a  random  variable.  Bowever,  these  j 

numerical  characteristics  do  not  reflect  all  the  specific  properties  of  a  distri-  | 
button  curve.  :| 

One  such  property  is  the  sfmetry  or  a^naetiy  of  a  distribution  curve  with  re-  t 
spect  to  the  axis  passing  thirau^  its  gravitational  center.  any  syea^trical  dis-  -1 

X 

tributlon,  an  exaiq)le  of  idiich  is  the  normal  lav  of  distribution,  evexy  odd  central  | 

moment  is  equal  to  zero;  this  foUovs  directly  from  (2.68).  Theimfore  che  sisq;>lest  | 

1 

of  the  odd  moments,  the  third  central  moment,  may  in  the  first  approximation  serve  1 
as  an  asymmetiy  characteristic  of  the  lav  of  distribution.  I 


The  third  central  moment 


(2.80) 


nay  ba  expressed  In  texms  of  the  first  three  moments  similarly  to  hov  the  dispersion 
is  expressed  in  (2.74)  in  terns  of  the  first  and  second  moments. 


Cubing  the  binomial  in  the  integrand  of  (2.80),  ve  find 

Af| = nij  ~  Sfliiifij 2/nJ . 


(2.81) 


In  mathematieal  statistics,  it  is  customazy  to  dxaracterize  the  asymmetry  of  a 


distribution  curve  ty  the  dimensionless  expression 

M  ' 

•  L _ 

KmI’. 

mhich  is  called  the  coefficient  of  asyasetry. 


(2.82) 


in  example  of  a^metrical  distribution  is  the  Rayleigh  distribution  (2.61). 
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« 

I  ' 


Ibe  third  Mnent  for  this  distribution  is  equal  to 


Therefore 


K -f -3.  .2a»4. V 

,=(— 

and,  oonsequenilj,  the  coefficient  of  asgneietry  of  the  Bajleich  lav  of  distribution 


is  equal,  in  accordance  vlth  (2.82).  xo 


k=2^j^.J^=0.63. 


(2.83) 


As  a  characteristic  of  the  anoothness  of  a  distribution  curve  about  its  node. 


there  is  c^>lo!7ed  the  dieensionless  coefficient  of  excess 


(2.84) 


For  c  nonul  lav  of  distribution 


•  €»-«F  m  ^ 


and.  since  «  o^.  according  to  (2.84) 

T=^-3=o/ 

Thus  the  nonul  distribution  curve  has  a  sero  coefficient  of  excess.  A  positive 
value  of  T  indicates  that  in  the  vicinity  of  the  node  the  distribution  curve  has  a 
hitler  and  sharper  peak  than  a  nonul  distribution  curve  vith  the  sane  gravitational 


center  md  diversion.  A  negative  value  of  the  coeffieient  of  excess  indicates  a 
flatter  peak  than  that  of  the  corre^nding  nonul  distribution  curve. 


Analogously  to  and  K^.  the  fourth  central  noaent  is  ejqpressed  in  tenu  of 
the  initial  nonents.  This  expression  has  the  fora  of 
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Af  4 = m4  —  4mjm,  -f-  6m,mJ  —  3mJ . 


(2.6i) 


Utlllcation  of  (2.85)  facilitates  the  computation  of  the  coefficient  of  excess. 

Let  ust  for  instance,  determine  the  coefficient  of  excess  for  the  Rayleigh  dis« 

tribution  (2.6l).  \h  find  first  the  magnitude 

••  <« 

m4=*ij  x‘e~^dx  =  83* 

and,  keeping  in  mind  the  values  obtained  for  and  m^  of  the  Rayleigh  dlstribu- 

tlon,  we  obtain  by  means  of  (2.85) 

The  coefficient  of  excess  is,  according  to  (2.^),  equal  to 

32  — 3it» 


^-3^ -0.3. 


•  (4-«P 

which  indicates  the  flatter  and  lo^er  character  of  the  Rayleigh  distribution  curve 

and  a  dis- 


in  comparison  to  a  normal  distribution  curve  with  a  mean  value  of  o’  ff 
perslon  of  2 


Let  ns  note  that  if  the  fom  of  the  distribution  function  is  given,  but  the 
values  of  the  parameters  are  unknown,  these  unknown  parameters  are  determined  by 
means  of  the  moments  of  distribution.  Thus,  for  instance,  both  parameters  of  the 
normal  law  of  distribution  are  determined  by  the  mean  value  and  dispersion.  If  the 
distribution  function  depends  on  a  large  number  of  parameters,  for  their  determi¬ 
ne  .^on  there  may  be  necessary  a  knowledge  of  the  value  of  the  coefficients  of  asym¬ 
metry  and  excess,  or  even  of  moments  of  a  higher  orderthan  the  foiirth.  However,  if 
the  form  of  the  distribution  fbnction  is  unknown,  the  knowledge  of  some  number  of 
c’istribution  moments  does  not,  generally  speaking,  provide  a  possibility  of  pre¬ 
cisely  deterr’nlng  the  unknown  distribution  function. 
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Numerical  Characteristics  of  Aggregate  of  Randcci  Variables. — Cp^Xatlon 

Coefficient 

Let  us  examine  the  aggregate  of  two  random  variables  ^  and  7^ »  the  two-dimen¬ 
sional  distribution  function  of  ^Ich  Is  equal  to  (2.3**) 

(2.66),  we  find  the  mean  value  of  each  of  these  variables* 

0*  MO* 

m,{5}=  Jxa>„(x)rfx=  J  ^xWi{x,y)dxd\/,  (2.86) 

«ih}— J>“'ia(y)<^y=  1  ^y^i{x,y)dxdy.  (2.86*) 

*-00  <^00  ""OO 

fiaploylng  a  geometrical  approach,  the  magnitudes  m^  (?)  Si  ft]  may  be  re- 

garded  as  the  coordinates  of  a  point  which  denotes  the  mean  value  of  the  position  of 
a  point  on  a  plane  with  the  random  coordinates  (§  ,^|^). 

Analogously,  the  dispersions  of  each  of  the  random  variables  can  be  found 
through  the  two-dimensional  function  WgCx,^) 

|5}  =  J  (X  —  mi  {5  })*  a»u  (x)  dx  = 


=  I  J(x-m,{E|)»it^,(x.y)dxdy. 

—00—0* 

«0 

Af,(nj=  J(y  — mi{tj})»ip,j(y)</y  = 

— «0 

«9  0» 

=  j  J(y-m,  {ti})*W8(x,y)dxdy. 


(2.87) 


(2.88) 


Let  us  find,  for  instance,  the  mean  values  and  the  dispersions  of  two  normally 
distribute  random  variables.  Bnploying  (2.57)  and  (2.58),  we  obtain  on  the  basis 
of  the  fomxtlas 


•  "»l  {5  j  m,  (tjj  ==0j.  Af.  }  =aj,  Af,  {t\}==4 . 


(2.89) 


Jn  this  manner  there  is  determined  the  meaning  of  four  (out  of  five)  parameters  of 
the  two-dimensional  distribution  function  (2.51)*  vd^ich  are  the  mean  values  and  the 


*  See  footnote  on  p.  83. 
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dispersions  of  each  of  the  two  random  variables  subject  to  the  normal  law  of  distri¬ 


bution. 


For  an  aggregate  of  two  random  variables,  there  is  possible,  besides  the  dis¬ 


persion  of  each  of  the  two  variables,  one  more  second  moment 


m 

Af„{S.ii}==  j  j(x  — m,  {£})(y~/n,{ti})tej(x.y)dxdv  = 

r'oT*  (2.90) 

o» 

idiich  is  called  the  mixed  second  moment  or  covariance  of  random  vp.:iables  ^  and/^ 
If  ^  and?^  are  independent,  then  ^ follows  from 


(2.90)  that 


r  * 

Ui=  J(Jf  — «!{?}) a'n(x)d*'  JCj'  — /nj{Tj})tt'j,(y)Jy=0, , 


since  each  of  the  integrals  turns  to  zero.  Therefore  the  values  of  a  mixed  second 


moment  may  serve  as  a  measure  of  dependence  between  two  random  variables.  More  fre¬ 


quently,  as  such  a  measure  there  is  employed  the  dimensionless  ratio 


p  (?T,} 

y  Af,  {IpfTfi}'  ’ 


(2.91) 


which  is  called  the  coefficient  of  correlation  (coupling)  ^sic/  between  the  random 
variables  ^  and  Tj  .  The  magnitude  of  the  ciafficient  of  correlation  is  always 
contained  within  the  limits  of  -1  ^  R  +1 •  Ihe  limiting  values  are . attained  only 
when  ^  and  are  linearly  dependent. 

If  the  random  variables  are  independent,  R  =  0.  The  converse  conclusion  as  to 
the  independence  of  ^  and  ?l  ^en  R  =  0  is  in  the  general  case  Invalid.  Two  random 
variables,  for  which  the  coefficient  of  correlation  is  equal  to  zero,  are  called  un- 


eorrelated.  Thus,  Independent  random  variables  are  always  uncorrelated,  but  not  the 


other  way  around. 


For  instance,  let^^ecos^and  ^ere  ^  is  a  random  variable  evenly 

distributed  between  0  and  2  if  ,  It  is  clear  thatC^  and  ^  are  de¬ 

pendent,  however,  their  covariance  (and,  consequently  their  coefficient  of  corrcla- 
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tlon)  is  equal  to  tero* 


'”i{^r5j}=^i{cost*sintJ  =— mi  jsin25j=0.  '| 

Let  us  find  the  mixed  second  moment  of  two  normally  distributed  random  variables,  i 

1 

Considering  (2.89),  ve  obtain  according  to  (2.51)  and  (2.90)  I 

-  *  /  I 

yi-r^  I  J(*, -«.)(*,-«, )exp{-2(i:b)X  I 

Extracting  the  square  from  the  expression  in  brackets  and  substituting  variables  | 

f 

a  =  -Jrrr-  (‘±~  \  r  \ _ *1  “  J 

■■■  1  ,  I 

ve  shall  have  ^  I 

fe"^a4-  I 

-4  -i  I 

_ _  <e  «•  M  yt  ^<5: 

,  *10,  V I  —  /■»  f  ~Tj  (*  ~T^  1 

i - 2S -  i 

from  which  we  find  ^ 

Af„=r8,3,.  ^2*92)  •  I 


from  which  we  find 


Considering  that  and 8^2  dispersions  of  random  variables,  and  keeping  in  mind 
the  definition  of  the  coefficient  of  correlation  (2,91),  we  find  that 


R  *  r 


Thus  the  fifth  parameter  r  of  a  two-dimensional  distribution  function (2,51  )is  the  coef¬ 
ficient  of  correlation  between  two  random  variables  which  are  subject  to  the  normal 
law  of  distzlbution. 

In  the  general  ease,  if  there  is  an  aggregate  of  n  random  variables  f 
tiie  distribution  function  of  which  is  ‘<^jj(?^»  the  mean  value  of  the 

random  variable  ^  t*  determined  by  the  formula 

*  If  15^(4  ^  0,  it  is  said  that  the  random  variables  ?  1  and  ^  2  “*® 

orthogonal. .  It  is  obvious  that  if  the  mean  values  of  the  random  variables  are  equal 
to  zero,  the  concepts  of  orthogonal  and  uncorrelated  random  values  coincide. 
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*‘04-  J-.-  ]x,w^{xi,...xjdxi..dx^, 

« rt« 


(2.93) 


The  central  second  noments  for  th<'  Indicated  aggregate  of  random  variables  are 


W  0. 


— «0  — « 


(2.94) 


When  j  ^  k  the  moment  Is  equal  to  the  dispersion  of  the  random  variable  ^ 
and  vhen  j  /  k.  the  moment  Is  the  covariance  of  the  random  vari  ibles  ^ 

and  ^  y  The  dimensionless  ratio 


A? 

'*  Wkk 


(2.95) 


is  called  the  coefficient  of  correlation  between  the  random  variables  ^  ^  and  ^  y 
For  an  aggregate  of  n  random  variables  there  can.  In  addition  to  moments  of  the 
first  and  second  orders,  be  defined  moments  of  any  order  t 


"**.*. . . .  (5,  -5, . . .  5, )  =  m,  {if . . .  e*" )  = 

•0  «»  «Q  ' 

=  J  J...  X„...X„)dx,dx^...dx„. 


(2.96) 


where  k^  +  kg  +...+  ^  and  kj(J  *  1.2....n)  may  be  any  whole  positive  member  (in¬ 

cluding  also  cero). 

As  a  rule,  the  applications  deal  only  with  real  random  variables.  However,  for 
the  simplification  of  certain  theoretical  investigations  it  is  on  occasion  conven¬ 
ient  to  examine  more  general,  complex  random  variables,  which  are  defined  by  the 
equality  =  ^  +  i  C  •  '‘here  and  are  real  random  variables. 

The  mean  value  of  a  complex  random  variable  is  a  complex  number,  equal  to 
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The  dispersion  of  a  complex  random  variable  with  a  zero  mean  is  equal  to 

* 

If  there  are  given  two  complex  random  variables  ^  ^  and  ^  2  means  1 

the  coefficient  of  correlation  between  them  is  determined  by  the  formula 


idiere 


Af„ = m,  {5i  •  tj}  =  Wi  { -f*  CjCj }  4*  ‘"*1  “  ^*1 1  • 


10,  Conditional  Distribution  Function* 

When  the  random  variables^and?|  are  dependent,  there  exists  the  conditional 
(a  posteriori)  probability  that  one  of  these  random  variables  lies  below  the  level 
of  y,  if  the  other  is  contained  within  the  limits  of  x<  ^  <  x^.  This  probability 
^x<5<’Xj  is  found  by  the  multiplication  rule 


P  ,  (tj  <y)= 

y)  P(x<5<x,) 


(2.97) 


If  there  is  given  a  two-dimensional  probability  density  w^Cj.z)  of  the  random  vari- 

ables,  it  is  possible,  bearing  in  mind  (2»28)  and  (2,31),  to  write  (2.97)  thus 

j  j  •t{x,y)dxdy 

- .  '  (2.98) 

J  J  •t(*.y)  dxdy 

Completing  in  (2.98)  the  limiting  transition  x^^— >  we  obtain  the  function 

(2.99) 


idilch  is  called  the  conditional  integral  distribution  function  of  the  random  variable 
7^  ,  under  the  condition  that  ?  =  x.  If  this  function  has  a  partial  derivative 
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•long  y 


!£^w=„(,/,). 


(2.100) 


this  derivative  is. called  the  conditional  probability  density,  or  the  conditional 
distribution  function  of  the  random  variable  under  the  condition  that  ^  ^  . 

Differentiating  the  right  part  of  (2,99)  with  respect  to  y,  we  find 

«(y/jK)= — _  »i(x.» 

?  ,  »iiW  •  (2.101) 

The  conditional  distribution  functions  have  all  the  properties  of  unconditional,  one- 
dimensional  functions.  In  particular,  it  follows  from  (2.101)  that 


Jw(y/x)dy=l. 


(2.102) 


Oonditional  numerical  characteristics  of  a  random  variable  may  also  be  intro¬ 
duced.  The  conditional  mean  value  of  the  random  variable  7^  ,  under  the  condition 
that  ^  ^  ,  is  called  the  quantity  ( 

'”uW=  ^yw{ylx)dy=z^ - .  j 

The  conditional  dispersion  of  the  random  variable  ^  ,  under  the  condition  that 

^  *  X ,  is  equal  to  1 

0» 

(’ll  —  J  (y — w  iylx)  dy=  | 

J  V-«fu^9t{x,y)dy  (2.104)  j 

I 

The  conditional  distribution  of  the  random  variable  ^  is  determined  analogously.  | 

4 

Thus  the  conditional  probability  density  under  the  condition  ^  »  jr  equal  to  | 

— .  I 

^m,(x.j0dx  ^  (2.105)  I 

If  the  random  variables  5  arid  ^  are  independent,  then  “  5^1  (?)  ‘^12^^^^ 

and,  as  can  be  seen  from  (2.101)  or  (2.105),  the  conditional  distribution  function  | 

-I 

coincides  with  the  unconditional  distribution  function  of  a  random  variable.  | 
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(2.105) 


Let  us  illustrate  the  formulas  of  the  present  section  by  the  exangjlc  of  an  ag¬ 
gregate  of  tvo  normally  distributed  random  variables.  Since  each  of  these  random 


variables  is  also  distributed  normally*  then 


(«-«■)* 

"ST 


JtB,(x,y)dy=w„(x)=-p=^e  ‘ 

and,  consequently,  the  conditional  probability  density  of  one  of  the  normally  dis¬ 
tributed  random  variables  ,  if  the  other  is  ^  *  x,  is  according  to  (2.101)  equal 
to  «(y/x)  = 

'  '  -i  I 

or,  after  reducing  the  similar  terms  in  the  e^nent,  ve  obtain 


idien  r  =  0,  which  corresponds  to  the  independence  of  the  random  variables,  the  con¬ 
ditional  distribution  function  (2.106)  tarns  to  a  one-dimensional  function  idiich 
corresponds  to  the  normal  law  of  distribution.  The  same  result  is  obtained  for  x  »  ap. 
but  with  the  diversion  multiplied  by  1  -  r^.  ; 

With  r— »1  in  accordance  with  (2.1?) 

'  '  N,  .!• 

:s 

which  corresponds  v)  a  linear  dependence  between  the  random  variables  f 

.  a— gi_g— g|  I 

^  *  I 

Fig.  15  shows  curves  of  the  conditional  distribution  functions  (2.106)  for  x  -  a^4r. 
'4  3.0;  and  for  several  values  of  the  parameter  r.  I 

*  ■*  a! 


■3  ’t  -I  9  I  3  3  9  Ht 

Pig.  15.  Conditional  distribution  functions  of  a  normal 
random  variable  under  the  condition  that  the  other,  de- 
.  pendent  normal  random  variable  takes  the  value  of  x  =  a^^  4 
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Let  us  also  find  the  mean  value  and  the  dispersion.  According  to  formula 
(2.103),  considering  (2.106)  ve  have 

,  ; _ L_  /■£=£! trftV 


s, 


(2.107) 


K2«  (I -/»)», 

=c,+r(x  — 0,)^. 

Wien_r  »  0  or  uhen  J  =  aj^  the  conditional  mean  coincides  with  the  unconditional. 

VIhen  X  *  a.]^  +  3  1  from  (2,10?)  there  follows  *12  +  3?  o'2*  quanti¬ 

ty  determines  the  abscissae  of  the  maxima  of  the  set  of  conditional  distribution 
function  curves  depicted  in  Fig.  I5. 

Vfe  find  the  conditional  dispersion  by  formula  (2.104),  considering  (2.106)  and 

(2.107) 


Af. 


.  .  2(1— f*)  I  «|  *11.  <> 

Xe  ^  ^  dy=<jl(l-~r^). 


(2.108) 


Th\  the  conditional  dispersion  does  not  depend  on  the  value  taken  by  the  ran¬ 
dom  variable  ^  ^  but  depends  only  on  the  correlation  coefficient  between  the  random 
variables  ^  and  ^  .  Vben  r  =  0  the  conditional  dispersion  coincides  with  the  un- 

eonditionil  *  1^2  * 

2 

11.  Measure  of  Uncertainty  Corresponding  to  a  law  of  Distribution. 

Together  with  the  nume;-lcal  diaracteristics  (moments  of  distribution)  examined 
above,  it  is  desirable  to  introduce  a  magnitude  which  would  characterise  the  degree 
of  uncertainty  created  by  the  various  laws  of  distribution. 

Hi  fact,  every  law  of  probability  distribution  describes  some  condition  of  un¬ 
certainty.  Ihus,  a  discrete  random  variable  nay  take  any  of  n  values,  and  ve  know 
only  the  probabilities  of  the  realisation  of  these  possible  values.  The  degree  of 
this  uncertainty  is  different  for  various  lavs  of  distribution.  For  instance,  when 
a  random  variable  may  take  only  two  discrete  values,  the  law  of  distribution 


■<'>={0: 


0,5  x=0, 
5  x=l 


(2.109) 
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contains  considerably  more  tmcertainty  than  the  law  of  distribution 


f,(x)  =  f*®  *  =  0.33. 

i0,l  x  =  2. 


(2,109*) 


It  is,  however,  not  difficult  to  calculate  that  the  nean  values  and  dispersions  of 
tvo  random  variables,  distributed  according  to  (2,109)  and  (2.109*),  are  equal  to 
each  other.  This  simple  exaDq>le  indicates  that  the  measure  of  uncertainty  corre¬ 
sponding  to  a  given  law  of  distribution  should  depend  only  on  the  magnitudes  of  the 
probabilities,  and  should  not  depend  on  the  concrete  numerical  values  idiich  a  random 
variable  nay  take.  Therefore,  in  introducing  the  indicated  measure,  it  is  necessaiy 
to  digress  from  the  nxuaerical  characteristics  of  random  events,  and  to  retura  to  an 
examination  only  of  their  qualitative  features,  as  was  done  in  Ch.  1. 

As  a  veiy  convenient  measure  of  imcertainty  for  the  discrete  ease,  there  nay 


serve  the  magnitude 


(2.110) 


idilch  by  analogy  with  certain  physical  magnitudes  is  called  the  "entropy"  of  the  law 
of  distribution.  Sx  fomula  (2.10)  ...Pjj  is  the  aggregate  of  the  proba¬ 

bilities  which  characterize  the  law,  under  discussion,  of  the  distribution  of  a  dis¬ 
crete  random  variable  (or  the  aggregate  of  the  probabilities  of  a  set  of  events). 


It  is  clear  that,  always. 


5;p.=<. 


From  the  definition  of  entropy  it  follows  that  H  ^  o  in  the  case,  and  only  in 
the  ease,  that  one  of  the  numbers  p^,  i2*’**?n  unity  (and,  consequently, 

the  others  are  equal  to  zero).  But  that  is  precisely  the  ease  when  all  uncertainty 
is  absent,  since  it  is  known  for  certain  which  event  must  be  realized.  For  all  the 
other  eases,  idxen  uncertainty  exists,  entropy  is  a  positive  magnitude. 

Of  all  the  laws  of  distribution  of  a  discrete  random  variable,  which  nay  take 
any  one  of  j)  possible  values,  the  gjreatest  uncertainty  is  possessed  by  the  uniform 
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Pk~~jr»  2, 


For  the  uniform  law  of  distribution 


(2.111) 


Obriously.  uhen  n  equiprobable  results  of  a  test  are  expected,  the  degree  of 
uncertainty  grows  with  the  increase  of  n.  The  entropy  of  tiie  uniform  law  of  proba¬ 
bility  distribution  is  equal  to  the  logarithm  of  this  number  n.  Therefv'tre  it  '  s 
sometimes  called  the  logarithmic  measure  of  uncertainty. 

In  the  general  case  of  n  expected  results  of  a  test,  some  will  be  more  probable, 
others  less  probable.  Hove  rer,  the  arbitrary  law  of  distribution  for  n  possible  re¬ 
sults  may  be  replaced  by  an  equiprobable  distribution  ^  n  possible  results,  having 


the  same  entropy 


'  /^(n)=]n/i)<]nn. 


For  a  continuous  unifora  distribution  w  (x)  =  »  S  <  x  <  fe.  the  entropy 

must  be  equal  to  the  logarithm  of  the  width  of  the  interval  of  b  -  a,  i.e.,  ln(b  -  a). 


This  magnitude  for  a  uniform  distribution  coincides  with  In 


wTxT 


.  It  is  possible 


to  find  the  average  statistical  value  of  In  in  the  general  case  of 

m 

— 

and  to  assume  it  to  be  the  measure  of  uncertainty  of  the  continuous  distribution 
(under  the  condition  that  the  cited  integral  converges).  In  this  manner  we  come  to 


the  following  definition  of  the  entropy  of  continxious  distribution 


^=—  Jw(Jt)lnu>(x)dx. 


(2.112) 


Analogously  to  the  discrete  case,  of  all  the  distribution  lavs  of  a  continuous 
random  variable,  the  possible  values  of  which  are  bounded  by  the  interval  of  (a,b), 
tha  maximum  entropy  -  ln(b  -  a)  is  possessed  ly  the  uiilform  law. 

Of  the  aggregate  of  all  the  distribution  functions  which  have  the  same  dispersion 
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of  normal  distribution  is  the  most  random,  i.e,,  possesses  the  maximum  entropy. 
The  entropy  of  normal  distribution  is  equal  to 

> 

00  (jr— a)’  .  ,  ’  '  ’ 


—00 

_ _  00  (jT— <!)•  00  (JT— <!)• 


=  In 

//=In(3K^). 


(2.113) 


Let  us  compare  the  magnitude  of  the  entropy  (2,113)  of  normal  distribution  with 

the  magnitude  of  Ihe  entropy  of  uniform  distribution,  given  the  same  dispersion  of 
2  2 

O'  •  For  a  dispersion  of  o-  to  correspond  to  uniform  distribution,  it  is  necessary 
that  the  possible  values  of  a  random  variable  be  bounded  by  the  interval  of  b  -  a  = 
so-V^  ^f.  formula  (3.88)  in  the  next  chapter^*  The  value  of  the  entropy  corre* 
spending  to  this  interval  is  equal  to  In  (trVlZ).  The  difference  between  the  entropy 
of  a  normal  and  of  a  uniform  distribution  with  equal  dispersions  is  equal  to 

In  {jV^)-\ti(iVn)  =  \n^ 

«  0,18  natural  units* 

Let  us  also  ooBumte  the  entropy  H.  of  harmonic  vibration  with  a  random  phase 
and  cooqMire  it  with  the  entropies  of  normal  and  uniform  distributions  which  have 
equal  dispersions  (cf .  Section  2,  Ch.  3) 

=  -  4- f  In  f— •  ;7=i=  Wy  = 

*  Jl'a*  — >*  V’  Va*-ytJ  ^ 

« 

T 

+■!■  f  In(acos?)</?=|nr4-lno— In2  =  ln 
-  •  • 

•More  frequently  the  base  of  logarithms  is  taken  as  equal  to  2  and,  consequently, 
entropy  is  measured  by  binary  units.  The  conversion  from  natural  units  to  binary 
units  is  made  by  the  formula 
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a2  2 

Assuming  now  —  *  cr  «  let  us  find  the  difference  between  the  t  z-opy  of  a  normal 


distribution  with  a  dispersion  of  and  and  also  between  the  entropy  of  uni¬ 
form  distribution  with  the  same  dispersion  and  H| 


In  (3  V2-it)  -  //,  =  In  (3  /2-e)  -  In  (-f  a  = 

=  In^2^  — j  natural  units 

in  (3  yV2)-H^  =  In  (3  J/T2  -  In  (-I*  3  J^2  )  = 

=  Infill) 

\  *  /  *  0,76  natural  units 


The  concept  of  entropy  can  be  expanded  to  the  case  of  multidimensional  laws  of 


distribution.  If  the  aggregate  of  random  variables  ^  ^2****^n  linked  by 

the  n-dimensional  distribution  function  the  entropy  of  this  dis¬ 


tribution  is 


00 


(2.114) 


liien  n  s  2 


H~—  J  Jw5(x,y)lnw,(x,y)rfxdy.  (2.114») 

If  4  1  *“^42  independent,  then  by  virtue  of  {Z,y>)  ^ 


oe  m 

j  J  J  »ii(*)wi»(y)llnwii(jf)  +  ln»i,(y)ld*dy= 

— OO— 00 

00  o» 

=  —  J»ii(*)Injr„(x)dx--  Jw„(y)Inaf„{y)i/y= 


(2.115) 


where  by  and  are  designated  the  entropies  of  the  distribution  functions  of  the 
independent  random  variables  4  ^  42* 


Thus  the  entropy  oi  the  joint  distribution  of  two  independent  random  variables 


is  eq*ral  to  the  sum  of  entropies  of  the  distribution  of  the  component  variables. 


If  and  4  2  are  dependent,  then,  introducing  a  conditional  distribution  func¬ 


tion  according  to  (2.101),  we  obtain 
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^  I  J*^^'^*»wnn«f„(x)4.1nw(y/x)|rfx</y 

and,  considering  (2.102).  ve  find  that 

m  y 

jm(ylx)lttw(ylx)dy. 

Ihe  integral 

=  J»(yWInw(y/x)dy  (2.116) 

is  called  the  conditional  entropy  of  the  randon  Tariable  ^  under  the  condition 
that  ^  ^  s  z.  BQ>loying  the  designation  of  (2.116).  it  is  possible  to  represent  the 
entropy  of  the  Joint  distribution  of  tuo  dependent  randoa  variables  in  the  form  of 


(2.117) 


If  ^  ^  and  $2  independent.  thenu»^/x)  =  w^2^)  fomala  (2.117)  turns  to 
fotwda  (2.115). 

Let  us  detereine  the  entropy  of  the  tiio>diHensional  nomal  lav  of  distribution. 
The  nagnitude  is  according  to  (2,113)  equal  to  *  1^  (<rj  iTi  7c  C  )._  aiil*  * 
ve  find  fros  (2.116),  utilising  (2.106) 


=ln/2«(I-r*)*|  +  -I.. 


(2.118) 

1h«i  the  entropy  of  a  tvo^dimnsional  nomal  distribution  is 


//=ta  (»,  K’Sw) + In  (3,  |r2^(I-r*‘> = In  (2=e»rt/ri;5). 
and.  considering  (2.108).  ve  obtain 

Il=ln(2=e3,»^). 


(2.119) 
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trtiere  Sx  is  the  conditional  dispersion  of  4  2*  in  the  ease  at  hand  does  not 

depend  on  what  value  is  taken  ,  but  depends  only  on  the  coefficient  of  cor¬ 
relation  between  ^  ^  and  ^  g* 

<  ln(<^£  a  H  ;  the  equality  sign 

y 

corresponds  to  r  =  0,  i.e«  when  ^  ^  and  ^  g  are  uncorrelated.  Ihe  inequality 
^/x  ^  ^s^3^1^3hed  here  for  a  special  case,  always  holds  true.  Therefore  it 

follows  from  (2.11?)  that 


It  can  be  seen  from  (2.118)  that 


(2.120) 


equality  being  attained  only  for  uncorrelated  (and,  consequently,  also  for  independ¬ 
ent)  random  variables.' 
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Chapter  in 


FUNCTIONS  OF  CONTINUOUS  RANDOM  VARIABLES 


1.  Functional  Transformations  of  Random  Variables 


In  practical  problems  it  is  very  often  necessary ,  on  the  basis  of  a  given  distri¬ 
bution  function  of  an  aggregate  of  random  variables  ^  ^2***'^n* 

to  determine  the  distribution  function  another  aggregate  of  ran¬ 

dan  variables  7^^ ,  ^2****^n»  '•^^ch  is  obtained  from  the  first  by  the  functional 


transfonnation 


^2  (^l*  Sj,  .  . .  5^), 


(3.1) 


where  f^,  C2****-n  given*  single-valued  and  continuous  functions. 

Let  us  first  examine  the  one-dimensional  case.  Let  there  be  given  the  distri¬ 
bution  function  Oi  (x)  of  the  random  variable  ^  ;  it  is  required  to  find  the  distri¬ 
bution  function  W(y)  of  the  random  variable  ^  ^  {i£f)»  Let  us  assume  that  there 

exists  the  inverse  function?  =  i.e.,  the  random  variables  ^  and?^  are 

linked  to  each  other  by  a  mutually  single-valued  relationship.  Then  from  the  fact 


jeo<5<JCo+</^. 

it  follows  for  certain  that 

yo<^<yo+<^y.  yo=f(*o) 


(3.2) 


(3.3) 


and*  conversely*  from  (3.3)  there  follows  (3.2).  Therefore  the  realization  proba¬ 
bilities  of  these  inequalities  are  equal  to  each  other.  The  probability  of  the  real¬ 
ization  of  inequality  (3.2)  is  equal  to  the  area  of  (Fig.  16)*  and  the  probability 
of  the  realization  of  inequality  (3.3)  is  equal  to  the  area  S„. 

Jl 


P-TS-981 1/V 


Fig.  16.  Functional  transformation  of  a  random  variable. 
VBLth  sufficiently  small  values  of  ^  and 

4  =  "(x)d2c,  ^  =  W(y)*, 

and  since  these  areas  must  equal  one  another,  ve  find  that 

W{y)=.w{x)~, 


i.e, ,  the  desired  distribution  function  of  the  random  variable  7jy  expressed  in  terms 
of  the  distribution  function  of  the  random  variable  ^  and  the  derivative  of  the  well- 
known  function^  *  ?($)♦  Since  0,w(x)  >0,  it  is  always  necessary  to  substi¬ 

tute  into  formula  (3»^)  the  absolute  value  of  the  derivative  ^  , 

If  the  function  ^  -  £(z)  is  such  that  its  inverse  function  x  =  is  not 

single-valued,  one  value  of  y  corresponds  to  several  branches  of  the  function  y  (y) . 
Let  us  denote  them  by  x^(y).  XgCy)*** 


Then  from  the  fact  that 


yo<^i<yo+</y. 


there  follows  one  of  the  mutually  exclusive  possibilities 


Xi<E<x,4-rfx,,  or  Xa<5<x2  +  dx2,  or  ... 


(3.5) 


(3.6) 


Etaplpying  the  rule  of  addition,  we  find  that  the  realization  probability  of  inequali¬ 
ty  (3.5)  must  equal  the  sum  of  the  realization  probabilities  of  each  of  the  inequali¬ 
ties  (3.6).  Each  of  these  probabilities  is  equal  respectively  to  the  areas  of  S 

and  S_  ,  S„  ,,,,(Fig.  17)  and,  consequently, 

“*1  ~?2 
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or  W(y)<fy’  =  +  ....  From  the  last  equality  we  obtain  the  de¬ 

sired  formula  for  the  distribution  function  W(y)  when  x  =  y  (y)  is  multivalued 

»'(y)=.'(<,)|*|+»(x,)|^'|+..,  (3.7) 

In  view  of  the  considerations  expressed  above*  the  absolute  values  of  the  derivative 
are  invariably  employed  in  (3.7) 


Lit. 

,  J 

\A 

1  *1 

*1  *1  ^ 

Fig.  1?.  Two-valued  inverse  transfonoation  of  random  variable 

It  is  not  difficult  to  expand  formula  (3.7)  to  the  two-dimensional  case  when  it  is 
necessary,  on  the  basis  of  the  two-dimensional  distribution  function  ^2^-1  *  -2^ 
the  random  variables  ^  ^  and  ^  determine  the  two-dimensional  distribution 
function  of  the  random  variables  and  which  are  linked  to  the  former  by  the 
definite  functional  relationship 


^)- 


If  the  inverse  transformation 


^2). 

^  =  ?2(ll.  %) 


(3.8) 


(3.9) 


is  not  single-valued  and  has  several  branches  from  the 

fact  that  point  A  with  the  coordinates  ^  ^ ,  7^^  is  situated  in  some  region  dS  (Fig. 
18),  it  follows  that  point  B  with  the  coordinates^^,  ^2  situated  either  in  the 
region  ds^,  or  in  the  region  ds2f  or....  Employing  the  rule  of  addition,  we  find 
that  the  probability  of  finding  point  A  in  the  region  ^  is  equal  to  the  sum  of  the 
probabilities  of  finding  point  B.in  each  of  the  regions  (k  =  1,2,...).  Each  of 
these  probabilities  is  equal  respectively  to  the  volumes  Vy  and  • . . .  (Fig. 

18),  and,  consequently. 
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h  -  hi* 


ys)</5  — a*,(x„,  JC2,)rfs,+a'2(x,2,  X22)rf52  4-... 


It  is  well  known  that  in  the  transition  from  the  variables  (jLjt  2^2^  ^  ^he  variables 
•  £2^  ratio  of  the  elementary  areas  and  dS  is  equal  to  the  so-called 

Jacobian  transformation  where^ 

■  £*1  ilt 

<fyi  dyt  _d(Xi.  xr)  «hareuDon  . 

«/s  dxt  dxi  (^(yi.yz)  *.  '  ‘♦O'l.ya) 

^  diTi 

Consequently ,  !  ' 


®^j(yi»yi) — I  d  (yi.  ya)  1”^ 


(3.10) 


If  the  correspondence  between  (  ^  mutually  single  valued, 

only  the  first  term  remains  of  equation  (3*10)» 


Fig.  18.  F\inctional  transformation  of  two  random  variables. 


In  the  general  ease,  if  the  Jacobian  of  the  transformation  from  the  random 

variables  ?  ^2’****  ^  n  ^  random  variables  ^  ^n  ta»wn, 

1^  ^  I 

Uyx'"dyA 


'CKi.  — >„) 


and  if  this  transformation  is  mutually  single-valued,  the  distribution  function  for  1 


7^  1*^2'***^n 

y2.  ...  y«)  =  |D|i»;,(x,,  xj,  ...  x„). 


(3.11) 
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2.  Elementaty  Transformations  of  One  Random  Variable 


Let  us  cite  several  examples  of  the  employment  of  formula  (3.7)  In  cases  where 
the  transformation  ^  =  ^(^  )  is  performed  by  means  of  the  simplest  elementary 
functions. 

The  linear  transformation  of  a  random  variable  ^  *  1.^  +  k  is  mutual^  sin^e- 
valued.  Therefore  in  accordance  with  (3.^)  we  have 


(3.12) 


Thus  in  the  linear  transfomatlon  of  a  random  variable  its  distribution  curve  is  ■ 
displaced  by  the  amount  b,  and  the  scales  of  the  coordinate  axes  change  a  times. 

In  the  quadratic  transformation  of  a  random  variable  7^  =  ,  every  value  of 

»diich  is  always  positive,  corresponds  to  tW5  values  of  the  random  variable  ^ 

Then  according  to  formula  (3.7)  we  find,  then  y>  0, 

And  so  the  distribution  function  of  the  square  of  a  random  variable  has  the  form  of 


117 A.) - ?  »(Ky)  +  u>(- V7) 

Yy  2 - when  y  >  0, 

W^)  =  0  when  y  <  0. 


(3.13) 


Let,  for  instance,  the  random  variable  4  he  normally  distributed  with  a  mean  value 
of  tero.  After  quadratic  transformation,  the  distribution  function  will,  in  accord¬ 


ance  with  (3.13)*  he  equal  to 


Y  2k  9 


(3.1^^) 


n^(y)=o.  y<o. 

Let  us,  finally,  examine  the  transformation  of  a  random  variable  by  means  of  the 
trigonometric  function  «  a  sin  w  ^  ,  Vlith  such  a  transformation,  to  each  of  the 
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possible  values  of  ^  contained  within  the  limits  of  from  -a  to  +  a  there  corre¬ 
sponds  an  infinite  number  of  values  ot^ 

fr  =  0,  rtl,  ±:2,  ... 

According  to  formula  (3.7)  we  find 

iy|<a 

“■/'-(?) '  0.15) 

W‘(y)  =  0.  ly|>a. 

Let,  for  instance,  the  random  variable  §  be  unifonaly  distributed  over  the  interval 
of  —  ^  ^  X  <“  ,i.e.,  o;  (x)  =  ^  whenjxj^  ^  and  O^enjxj  :> 

Then  in  the  infinite  summation  (3*15) »  when  k  =  6  and  k  =  1  only  two  terms  will  dif¬ 
fer  from  zero. 

The  distribution  function  of  the  random  variable  7?  ”  &•  sin  a;  ^wUl  take  the 


following  form  (Fig.  19): 

^  (3.16) 

tt‘(y)=0,  lyl>fl. 

Function  (3.16)  may  be  regarded  as  the  distribution  function  of  the  values  of  a 
random-phase  sinusoid,  i.e.,  the  position^  of  a  point,  moving  in  accordance  with 
haraonic  law  in  the  random  moment  of  tine  under  the  condition  that  any  of  the 

moments  within  the  limits  of  one  period  ^  are  equiprobable.  Let  us  note  that 
this  distritwtion  depends  only  on  the  an^jlitude  £  of  the  harmonic  vibrations  of  the 
point  and  does  not  depend  on  the  period  X. 


Pig.  19.  Probability  density  of  values  of  a  sinusoid  with  a  random  phase. 
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i*^  'i^v«^  V^T"  { : 


1 


Bearing  in  mind  the  relationship  (2,34),  it  is  nov  possible  to  find  the  distribution 
function  of  the  one  random  variable  * 


CO 

«^i(y2)=  J  tt'tCyi.  ys)‘/yi- 


(3.21) 


Substituting  (3*20)  into  (3*21)  and  performing  the  replacement  u  =  ^^  ^yt^'  ^ 
tain 


tw  /  V  r  «*  {“•?»[/!  {'O.yil) 
ttf'i  (y^)  =  \  T-5— ^ - —  du. 

*'  '5x;^*  1“«  »alfi(«).  >j1} 


(3.22) 


Let  7^^  s  ^  ,  i.e.,  f^(x)  =  x;  then  it  follows  from  (3.22)  that 


m 

r.(y,)=  r  g»J«.ta(«.ya)i  rf„. 


(3.23) 


Formula  (3.23)  makes  it  possiULe  to  determine  the  distribution  function  W^(y2^ 
of  the  random  variable  T^g  obtained  as  the  result  of  a  functional  transformation  of 
the  two  random  variables  ^  ^  and  ^  2  ♦  joint  density  of  whose  distribution  is 
equal  to  d)  (x  ,  x.).  From  this  formula  there  are  obtained  as  special  cases,  the 

4  “1 

distribution  functions  of  the  sum,  difference,  product  and  quotient  of  the  two  ran¬ 
dom  variables. 

Thus,  for  "^2  =  ^  ^  ?2  ~  ^  »  *2  ^2  ”  5l  ^3-23)  we 

obtain  the  distribution  function  of  the  sum  of  two  random  variables 

ee 

(yi)  =  j  ^2  (".  y2 — «)  dit.  (3^ 24) 

Analogously  for  the  difference  of  twa  random  variables 

(y2)=  J  a-j(H.  >'2 -r «)</«.  (3.24* ) 

^  ^2  *  ^1*^2*  ~  ^1*  ^2  ~  U.23)  ve  obtain  the  dis¬ 

tribution  function  of  the  product  of  twa  random  variables 


i 

?  ■' 
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(3.25) 


» >>  I 

f I 


va 


•a  0  «ff 


Fig.  20.  Integral  distribution  function  of  values  of  a  sinusoid  vith  a 

random  phase. 


The  Integral  distribution  function  corresponding  to  (3*16)  is  equal  (Fig.  20) 


^(y)  =  4-+varcs5n-^’-.  |yKa, 

F{y)  =  0, 


(3.17) 


..  Arithmetical  Operations  on  Tvro  Random  Variables, 


Let  us  examine  the  following  special  case  of  transformation  (3.8); 


■’ll  — /i  (^i). 


(3.18) 


under  the  condition  that  the  reciprocal  functions  ^  ^  s  ^  ^^2» 

^  g)  =  single-valued.  The  Jacobian  of  this  transformation  is  equal 


f  0 

<>(>!.  >'«)  __ 

'  dft  dft  (tXi’dx'i- 


dxi  itx. 


(3.19) 


The  two-dimensional  distribution  function  of  the  random  variables 


7^1  and 


is  in  accordance  with  (3.10)  equal  to 


W) 


(3.20) 
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m'  : 


U.  ! 


■'"'S 


Analogously,  the  quotient  of  dividing  one  random  variable  by  another  distribu¬ 
tion  function  is  equal  to 

m 

H^i(yj)=  J  (3.25*) 

for  independent  random  variables,  in  place  of  **^2  ?  *f  2  *  y2Junder 

the  sign  of  the  integral  in  formulas  (3.24)  and  (3.25),  there  should  be  placed  the 
product  of  the  distribution  functions  of  each  of  the 

random  variables.  Thus,  the  distribution  function  of  the  sum  of  independent  random 
variables  is  equal  to 

'•^1  (y2)==  J  (//)!£•, j (y.,~ti)du.  (3,26) 

The  integral  in  the  right  part  of  (3*26),  is  called  the  convolution  of  functions 


and  03^2* 


Let  us,  for  instance,  examine  what- form  will  be  taken  by  the  distribution  func¬ 
tion  of  the  sum  of  a  normally  distributed  random  variable  with  a  zero  mean,  and  the 
value  of  a  sinusoid  with  a  random  phase.  For  this  it  is  necessary,  in  accordance 
with  (3.26),  to  form  a  convolution  of  the  distribution  functions  (2.14)  and  (3.16): 


a  (y-K)* 

;»j»  uu 


I  I  r - aJi  du 

—  r  ‘  ”  a* 

y»  +1  o»P*  yop 

tc  I  2«*  J  Y  I  — V* 


-I 


Computation  of  the  integral  leads  to  an  expression  in  the  form  of  a  sum  of  the 
products  of  Bessel  functions* 


ya 

e  e 


N— 1 


♦  Footnote,  see  p.  95 
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■a 


"i 

'4 

4 


I 


1 


4 


4 

'i 


The  curve  of  this  distribution  for(^}^  =  j  is  shown  in  Pig.  21. 


-2  -if  -(I  0  i*  Q$  ^  0  20 


Fig.  21.  Distribution  of  the  sun  of  a  normal  random  variable  and 
a  sinusoid  with  a  random  phase. 

As  a  second  example  we  shall  examine  the  distribution  function  of  two  independ¬ 
ent  random  variables,  uniformly  distributed  along  the  segment  (a,^.  The  probability 

density  of  each  of  the  random  variables  is  equal  to  __J!: —  ,  if  j  X  ^  and  is 

-H  •  & 

equal  to  aero  outside  the  indicated  segment.  Etaploying  (3.25),  we  find  the  desired 
distribution  function  of  the  product  (Fig.  22): 


Ja — a' b — a  u  '  a  '  b 


L 

ir,(y)=0,  y<c*.  y>6» 


*  For  this  it  sufficient  to  substitute  the  integration  variable,  v  =  sin  t,  and  then 
to  employ  the  expansions  known  from  the  theory  of  Bessel  functions 


fli 

f*  ^  -  /,  (f)  +  2  J]  (c)  cot  mi. 
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li 


In  (3*27)  the  prlnaxy  value  of  the  arc  tangent  is  employed.  This  transformation 
is  mutually  single^valued,  with  only  the  positive  values  possiblev  the  pos¬ 

sible  values  of  the  random  variable  ^  being  confined  within  the  limits  of  from 
0  to  2  ,  Geometrically,  transformation  (3*27)  signifies  a  transition  from  the 

random  cartesian  coordinates  ^2)  nf  a  point,  to  its  random  polar  coordinates: 

the  length  of  the  radius  vector  ^  ^ ,  and  the  angle  of  inclination  of  this  vector  to 


the  abscissa. 

The  transformation  inverse  to  (3*27)  has  the  form  of 


5,  =  1IJ,COStlj, 
?j=i|isimi2. 


(3.28) 


Lot  there  be  given  the  two-dimensional  distribution  function  of  random  cartesian 
coordinates  Tr2(x^,  it  is  required  to  find  the  joint  distribution  function  of 

random  polar  coordinates  wj^Cp  •  ^ ).  Since  the  Jacobian  of  transformation  (3*28) 
from  the  variables  to  the  variables  ^  ,  y)  is  equal  to 


— psin<p  _ 

^  sin  ?  f  cos  9 


by  ent -l.-ylng  formula  (3.10)  we  obtain 

W^*(P.  ?)=pa'2('>f|.  ^j)=pa',(pcos9,  psin9)  ’ 

(»>0,  0<9<2::. 


(3.29) 


From  (3.29)  we  also  find  the  one-dimensional  distribution  functions  of  the 
length  "W  ^  and  angle  of  inclination  ^  2  radius  vector  to  be 

1*^it(?)=pJf<'2(pcos9,  psin9)d9,  p>0, 


(3.30) 


fl^ll(p)  =  0.  p<0; 


m 

*^«*(T)=Jp“'j(pc‘>s?.  p  sin  9)  dp,  0  <  9  =4  2i:, 


(3.31) 


9<0.  9>2r. 
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>i 


I 


d|: 


i 


4 

1 


It  is  not  difficult  to  generalise  formula  (3.29)  for  the  case  of  two  points  on 
a  plane,  idiose  random  cartesian  coordinates  are  dependent  and  characterised  by  the 
four-dimensional  distribution  function  W  ,  x^,  Xy  x^).  The  transition  to 
lengths  and  inclination  angles  of  radii-vectors  is  accomplished  by  means  of  the  trans¬ 


formation 


X,==p,COSf,:  *i  =  PjCOS?2, 

^=p,sin9,:  X4=p,sin(p2. 


(3.32) 


The  Jacobian  of  transformation  (3.32)  is  equal  to 

cos  9,  —PiSinfi  0  0 

sin?!  Pi  cos 9,  0  0  ^ 

J(Pi.  fi.fl>  fi)  0  0  cos  9,  —  pjsin9j 

0  0  sin  9}  P]C0S92 

Boploying  formula  (3.11),  we  find  the  four-dimensional  distribution  of  lengths 
and  inclination  angles  of  the  radii  vectors  of  the  two  points 

p,sln?„  PjCos?,.  p,sin92), 

Pl>0.  Pi>0.  0«49,<2n.  0<92<2t:. 

From  (3.33)  it  is  not  difficult  to  obtain  the  two-dimensional  distribution 
function  of  the  lengths  and  of  the  angles  of  inclination  1’  ^2^ 

of  the  indicated  radii  vectors 


W^«(Pi.  P2)=^ 


==Pip2jJ«'«(PiCo^fi.  Pi  sin  9,.  pjcos?*,  Pa  sin  9,)  d9,(/92. 


(3.3^0 


Pi>0.  P2>0. 


®^2l(Pl.  Pt)  =  Of  Pl<0,  p2<0, 

o,~  ®^22(?l»  T2)  = 

OiOD 

=JJpiP2“’4(PiC0S9,,  Pisin9i,  P2COS92,  Pasin92)dp,</p2. 

0<9|<2)c,  0<92<2ic, 

n^22(?i.  fi)  =  0,  9,<0,  9i>2it,  92  <0,  92>2ic. 


(3.35) 
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Li  the  general  ease,  the  transformation  from  a  2n-dlmenslonal  distribution  of 
the  random  coordinates  of  n  points  to  a  !te-dlmensional  distribution  function  of  the 
lengths  and  inclination  angles  of  the  radii  vectors  of  these  points  is  made  by  the 
formula 


fi.  ...  P«.  ?J=Pi...p,a»2<,{p|Cos(p„  ...  p,sin«p„) 
Pi>0.  ...p,>0,  0<^,<2r.  ...0<®  <2*. 


(3.36) 


special  eases  of  vhich,  vhen  ji~  1  and  n.=  2,  are  formulas  (3*29)  and  (3*33)* 

Generalised  Rayleigh  Distribution  Function 

Let  us  apply  the  results  of  the  preceding  section  to  find  the  distribution 
function  of  the  length  of  the  radius  vector  of  a  point,  the  coordinates  of  vhlch  are 
independent  and  normally  distributed  with  the  parameters  (a,  o-)  and  (b,  o-)  respec¬ 


tively. 


In  accordance  with  (3.30) 


co$  r-o)»  (f  sin  f— 6)» 

J»»  1  ~  ?•> 


dip,  p>0, 


or  after  elementary  algebraic  transformations,  we  obtain 


0 


p>0. 


idiere  0  =  arc  tan  y  , 


(3.37) 


The  integral  in  the  right  side  of  (3.37)  is,  through  the  substitution  of  y  — 
0  *  U  and  the  introduction  of  the  designation  0^=  ^  .  reduced  to  a  sero- 


order  Bessel  function  of  an  imaginaiy  argument 


a_4  -//Jit  *o.« 
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Thus,  the  desired  distribution  function  is  equal  to 

_H+i* 

(3^38) 

jr(p)=o.  p<o. 

Distribution  function  (3*38)  deteraines  the  probability  of  finding  two  independent 
random  variables,  distributed  according  to  the  normal  lav,  in  the  ring  (pi  f  +  ) 

vbose  center  lies  at  the  origin  of  the  coordinates.  This  function  generalizes  the 
Rayleigh  law  of  distribution  ^f.  (2.6lJ7>  idiich  constitutes  a  special  case  of  (3.38) 
when  a  *  Jfe  =  0,  Therefore  this  function  may  be  called  the  generalized  Rayleigh 
distribution  function.  The  ctirves  of  function  (3.38)  for  the  values  of  5=  =0;  1; 

2;  3;  5  are  shown  in  Fig.  24.  The  curves  of  the  Integral  distribution  law  correspond 
Ing  to  (3.38)  are  shown  in  logarithmic  scale  on  Fig.  25.  These  curves  were  obtained 
through  numerical  integration  of  the  generalized  Rayleigh  function  [?]  • 


If  ^  is  small,  the  generalized  Rayleigh  distribution  function  differs  little 
from  (2.61),  the  correction  being  obtainable  through  the  expansion  of  the  Bessel 
function  into  an  exponential  series.  Sometimes  it  is  possible  to  limit  one*s  self 
to  only  the  first  tw  terms  of  this  resolution  and  then 


r(p)=-J- 


0.39) 
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p  ^  p  ^  ^ 

For  Talues  of  "jr  <  •^formula  (3*39)  is  sufficiently  precise,  and  idxen 

precision  becomes  unsatisfactory.  But  if  ^  1  *  probability  density  for 

y  becomes  so  small  that  it  is  generally  disregarded  in  practice. 


Fig.  25.  Integral  Generalized  Rayleigh  Distribution  Function 


If  is  large,  then  in  expression  (3.38)  the  Bessel  function  may  be  replaced 


by  its  asymptotic  expansion 

'»<'>'•  Ffe('  +  ^+-)- 

Then 

r(p)-  -Je 

or  , 


£i 


.i 


« 

¥ 


i 


(3.40) 


Frcm  formula  (3*40)  it  follows  that,  with  accuracy  equal  to  the  correction  factor  of 
(■  •  ^)fT  ,  the  generalized  Rayleigh  distribution  law  turns  in  this  case 
into  the  normal  law  of  distribution  with  the  parameters  of «  and  o'  *  ^  ^  ^ 

~  I  is  not  large,  the  correction  of  the  normal  law  is  insignificant, 

IP  oc  t 

~  —  -p  j  •  although  the  correction  is  substantial,  the 
probabiliV  density  at  these  values  becomes  insignificantly  small. 

Let  us  determine  the  numerical  characteristics  of  a  random  variable  distributed 
according  to  the  generalized  Rayleigh  law.  In  accordance  with  (2.64)  the  k-th 
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initial  moment  of  distribution  (3^38)  is  equal  to 


■t*  0» 


nt 


.=  '.T'  ‘■'" 


As  a  result  of  computing  the  integral  ^f.  for  instance,  Vatson,  G.  N.  Teoriya 
besselevykh  funktsLy  (Theoiy  of  Bessel  Functions)^ Moskva,  For.  Lit.  Pub.  Hse.,  19^9^ 
(i.e.,  Vfatson,  G.  N.,  "A  Treatise  on  the  Theoiy  of  Bessel  Functions",  N-.f.  1944^7 
ve  obtain 


tn 


^  =  (2«i)‘  r^l  +  -Y)iFi[~-T<  ••  -•]?)• 


(3.41) 


vhere  ^F^-is  a  degenerate  hypergeometric  function,  the  basic  properties  of  which  are 
cited  in  Appendix  VI. 

The  mean  value  _m^  of  a  random  variable  distributed  according  to  the  generalised 
Rayleigh  lav  is 

•,»,=o|^2r(4),f,(-4,  I, 

+  23‘)'»(t5-)+  (3.'t2) 

From  (3.42)  as  a  special  case  with  <x=  O  we  obtain  formula  (2.79)  for  the  mean 
value  corresponding  to  the  Rayleigh  distribution, 

Biiq)loying  (3.41 )  it  is  also  not  difficult  xo  write  the  expressions  for  initial 
moments  of  the  2-nd  and  y-rd  orders, 

(3.43) 


m. 


,=33*}/’ (75r)  + 


(3.44) 


special  cases  of  idiich,  idien  <<  =  0,  are  the  corresponding  formulas  of  ^e^ibn  8jCh.2* 
Considering  (2.74),  it  is  likewise  not  difficult  to  obtain  from  (3.42)  and  (3.43) 
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£3 


m 


the  expression  for  the  dispersion  of  a  random  variable  vhich  is  distributed  according 
to  the  generalized  Rayleigh  lav. 


If  ot  » O'  ,  then,  employing  the  previously  cited  a^ptotic  resolution  of  the 
Bessel  function,  we  find  that 


(3.45) 


(3.46) 


The  first  items  in  formulas  (3*45)  and  (3*46)  are  respectively  the  mean  value  and  the 
dispersion  of  the  limiting  normal  law  of  distribution,  and  the  second  items  provide 
a  correction  which  diminishes  with  the  growth  of  . 

Substituting  (3*42),  (3*43)  and  (3*44)  in  (2,81),  we  find  the  central  moment  of 
the  third  order,  after  which,  considering  (2,82),  it  is  possible  to  compute  the  coef- 

OC 

fieient  of  asymmetry,  which  will  depend  only  on  ,  Results  of  the  computation  of 
the  coefficient  of  asymmetry  k,  for  the  curves  depicted  in  Fig,  24,  are  cited  in 
Table  3.  _ 


4 


Table  3 


0 

1 

2 

3 

5 

k 

0.63 

0.43 

0.24 

0.07 

0.015 

The  obtained  data  make  it  possible  to  Judge  the  extent  to  idiich  the  cv/ef fieient 
of  asynmetzy  Is  sensitive  to  changes  in  the  fora  of  the  curve.  From  a  comparison  of 
Fig.  24  with  the  data  of  Table  3*  it  is  evident  that  when  0.07,  i.e.,  with  a 
diminution  in  the  coefficient  of  asymmetry  of  about  10  times  in  comparison  to  the 
most  asymmetrical  Rayleigh  distribution  ( <^»  0),  the  distribution  curve  becomes 
practically  symmetrical  and  is  in  sufficient  proximity  to  the  curve  of  the  normal 
lav  of  distribution. 
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I 

'J 

i 


Let  there  be  given  the  distribution  function  6i)  (x)  of  the  random  variable  y  ; 
it  is  required  to  find  the  mean  value  of  the  random  variable  7^  =  /(^ ).  If  the  in- 
verse  fxmction^  =  (^)  is  slngle>valuedt  then,  employing  (3»^)t  we  obtain 


m, 


or 


00  00 

—00  —00 

00 

"»iH==  ^fix)w(x)dx. 
—00 


(3.47) 


It  can  be  demonstrated  that  formula  (3*4?)  of  the  mean  value  of  a  function  of  a  ran¬ 
dom  variable  is  valid  for  any  continuous  function  f(x). 

I>t77  -  Then  from  (3.4?)  it  follows  that 


w,  {?*}=  ^x*w{x)dx=:m^  {«}. 


(3.'*8) 


i.e*,  the  k>th  moment  of  distribution  may  be  treated  as  the  mean  value  of  the  k-th 
degree  of  the  random  variable. 

Analogously, 


Af*  {' }  ==  m*  {;  -  m^^  =  m,  {(;  -  m,)*}. 


(3.49) 


Since  the  mean  value  of  a  constant  is  equal  to  the  constant  itself,  then  from 
(3.49)  it  follows  for  the  special  case  of  ^  «  c  that  the  central  moment  of  a  con¬ 
stant  of  any  order  is  equal  to  zero. 

b  the  general  case,  when  there  is  given  a  distribution  function  of  the  aggre¬ 
gate  of  ^  ^ » •  •  •  ^  jj  random  variables  and  it  is  required  to  find  the  value  of  X  =  f 
(^  we  have 

0>  0* 

J  ...  ...rfv  (3.50) 

-00  — 3> 

"tiines 


F-TS-9811/V 


104 


.^T  yjftl/ii  '^J' 


The  special  case  of  formula  (3*50)  for  jl-  2  may  be  employed  for  finding  the 

numerical  characteristics  of  a  sum  and  of  a  product  of  random  variables* 

The  mean  value  of  a  sum  (or  difference)  of  random  variables  is 

«0  00 


m 


1  {«!  —  ^}  —  J  J  (Jfl  It  Xjj)  £t-j  (jf„  Jfj)  dx^dXi  = 
—00 

00  00 

=  J  Xi)dx^dx.^:^:. 

— OO  —00 

40  OO 

~  J  X2)dxidx.^, 


— »— 3® 


and,  considering  (2.86)  and  (2.86*),  we  find  that 


/«,  ri:  ^  iJ:  m ,  {‘cj} . 

Prom  formula  (3«51)  it  follows  that 


(3.51) 


m, 


(3.52) 

*-l  k-l 

Thus,  the  mean  value  of  a  sum  of  random  variables  is  always  equal  to  the  sum  of  the 
mean  values  of  the  terns. 

The  mean  value  of  the  product  of  random  variables  ^  ^2  ^ 

(2.90)  equal  to 

00  00 

/«1  {;,  -l^j  =  J  J JC,  (.X X2)  (:x,,lx2  = 

— 00— » 

If  4  ^  and  4  2  “’®  independent,  then 

m,  {5, . 5, }  =  m,  (3.53.) 

It  follows  from  formula  (3.53*)  that  for  mutually  independent  random  variables 


(3.53) 


n  n 

”''{n*.)=n»'i{y- 


0.54) 


Thus  the  mean  value  of  a  product  of  mutually  independent  random  variables  is  equal 
to  the  product  of  the  mean  values  of  the  factors. 
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r-TT' ^'^7^^''^rJ■;>c‘"'7*;■'^%  i.  . 


The  folloiring  rule  is  a  consequence  of  (3*53*)*  the  constant  factor  5.  may  be 
brought  out  beyond  the  sign  of  the  mean  value 

Let  us  now  examine  the  disj  srsion  of  the  sun  (or  difference)  of  tw>  random  vari¬ 
ables.  Bnploying  (3*^9)  and  (3-52),  we  find  that 

Afj  {«,  It; «j}  =  m,  {('i  t  --  m,  rt  5.^} )2j  — 

=  /«,  {((',  -  m,  -  (I,  -  -  m,  = 

and,  considering  (2.90),  we  obtain 

M,  {?,  It  ^}  =  A/.,  {!,}  -f.  Af,  {;,)  It  2AI,J  .*4.  (3.56) 

If  0  .  and  4  ,  are  independent,  their  mixed  moment  M  =0,  and  then 

I  £.  -^2 

AI,  |;,  t .  5,j  =  M,  |;,|  J-  Af,  1:4,  (3.57) 

i.e,,  the  dispersion  of-  the  sun  or  difference  of  two  independent  random  variables  is 
equal  to  the  sum  of  the  dispersions  of  these  variables.  Prom  formula  (3.57)  it 
follows,  that  for  mutually  independent  random  variables 


■I 


•'I 

.A' 

'4 


/if 


■Pi 

t 

.1 

X 


*->  *-l 


(3.58) 


Thus  the  dispersion  of  an  algebraic  sum  of  mutually  independent  random  variables  is 
equid  to  the  sum  of  the  dispersions  of  the  terms.  The  dispersion  of  a  product  of 
independent  random  variables  is  equal  to 


Afj  {5,  -E,}  =  m,  {(E,^  -  m,  {I, }  -m,  {5,} ]* j  =  Af j  )  •  Mj  {E,}  + 

+mJ{5,|Afa{Ej|+m*{y.Afa{5,}. 


(3.59) 


Expressing  dispersions  in  terms  of  initial  moments  according  to  formula  (2.74), 
after  elementaiy  transformations  we  obtain  from  (3.59) 


Afa {ErEaj  =/na  {E,}.ma  {5,j  - mj  {?,} -m*  {5,}. 


(3.59*) 
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Formula  (3.59')  g®neralizes  formula  (2.74)  for  the  two-dimensional  case. 

Since  the  dispersion  of  a  constant  is  equal  to  zero,  from  (3*59)  there  follows 
the  rule:  a  constant  factor  may  be  brought  out  beyond  the  sign  of  dispersion,  if 
this  factor  is  squared  in  such  a  case 

Af,{d}=c»Af,{5}.  (3.60) 


According  to 
^  —  m, 


this  rule,  for  instance,  the  dispersion  of  the  normalized  deviation 
of  the  random  variable  ^  is  always  equal  to  lunity. 


7.  The  Characteristic  Function 


In  the  preceding  section  there  has  been  presented  the  general  formula  (3*47) 
for  the  mean  value  of  the  random  variable  ^  ,  :diich  is  obtained  ty  means  of  the  func¬ 
tional  transformation  of  the  random  variable  ,  the  distribution  function  of 
which  is  equal  to  4^ (x).  In  subsequent  applications  an  important  role  will  be  played 
by  one  special  form  of  this  transformation,  namely 

(3.61) 

where  v  is  an  arbitrary,  real  parameter.  The  mean  value  of  tl:e  random  variable 
is  called  the  characteristic  function  of  the  random  variable  ^  or  the  characteristic 
function  of  a  given  probability  distribution.  In  accordance  with  for«»ula  (3.47)  the 
characteristic  function  of  the  random  variable  ^  is  equal  to 


(3.62) 


Since  j  e—  |  ^  Integral  (3.62)  converges  at  all  real  values  of  y  for  all  dis¬ 

tribution  functions  W  (x).  Therefore  a  characteristic  function  may  be  determined  for 
any  random  variable. 

Let  us  formulate  the  basic  properties  of  a  characteristic  function, 

1.  From  the  definition  it  follows  that 


I 


i 

f'P 

I 


!  1 


! 


I 


\ 


1 


j 

4 


i 

i 
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O' 


'im 


®(0)=  Ja»U)<^>^=l  «|©(P)|<0(O)=1. 


(3.63) 


2.  If  (-w  )  is  the  characteristic  fimction  of  the  random  variable  the 
characteristic  function  ('*^)  random  variable  ^  g*  ^<hich  is  obtained  ly 

oceans  of  the  linear  transformation  equal  to 


^4 

A 

4 

'I 


I' 


6„(tf)  =  J  a>  (jt)  dx  =  e„  (at/)  e'^ 


(3.64) 


7  •£ 

'  A 

<; 
5 


3.  If  there  exists  a  k-th  irdtial  moment  of  distribution  of  the  random  variable  ■ 


c  .? 

V  *  the  characteristic  function  of  this  variable  has  a  derivative  of  the  k.-th  order,  1 

vith 


<<*«  (o)  •> 

rft»»  ”■* 


J,*. 


'e'"a/(x)(/x. 


vherefrora 


(3.65) 


Thus  the  initial  moments  of  distribution  differ  only  the  factor  iK  from  the  value  | 

i 

of  the  derivatives  of  the  characteristic  function  when  v  =  0.  Ffom  (3.65),  in  the 

~  i 

special  case  of  k  =  1 ,  we  obtain  the  expression  for  the  mean  value  J 


«i{M=T®'(0)- 


(3.66) 

If  moments  of  every  order  exist,  then,  as  fellows  from  (3.65),  the  character¬ 
istic  function  may  be  represented  by  a  KacLaurin  series 


0.67) 


•-1 


4.  Central  moments  of  distribution  are  linked  by  simple  relationships  to  the 
derivatives  of  the  logarithm  of  the  characteristic  function.  Let  us  assume  ^ (v)  = 
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®ln  0  (v);  then 


A"  (v\ = 

T  «*(«)  ' 


and,  considering  (3«63)  and  (3.65)  we  obtain 

^"(0)=e"(0)-(e'(0)l*= 

=  -m,{5j4-/nJ  {5j  =  -Af,{{j. 

i.e.,  the  dispersion  of  a  random  variable  is  equal  to 

Af,{5}  =  -f'(0). 

Analogously  it  is  possible  to  obtain  the  formulas 


(3.68) 


Afj  S=  _  (0),  Mt  =  (0)  4-  ZMl , 

from  which  also  follow  the  expressions  for  the  coefficients  of  a^mmetry  and  excess 


_  f"(0)  _ 

ir(0)P 


(3.69) 


The  k^th  order  derivative  of  the  logarithm  of  a  characteristic  function  at  the 
point  of  V  =  0 ,  multiplied  by  i^,  is  called  the  k-  th  order  cumulant  or  semi-invariant 
of  a  random  variable.  As  can  be  seen  from  the  foraulas  presented,  a  knowledge  of  the 
semi-invariants  of  the  first  four  orders  makes  it  possible  to  find  veiy  simply  the 
mean  value,  the  dispersion,  and  the  coefficients  of  asymmetry  and  excess. 

5.  Let  ^  ^  and  ^2  ^  independent  random  variables,  the  characteristic  func¬ 
tions  of  sdiich  are  equal  respectively  to  (v)  and  (▼).  We  are  to  find  the 
characteristic  function  ©  (v)  of  the  sum  ^  +  *^2  of  these  random  variables. 

Since,  together  with  ^  ^  and  ^2  »  5^^  ^  *nd  e— are  also  Independent,  ty  em¬ 
ploying  the  property  (3. 53') of  the  mean  value  of  a  product  of  independent  random 


variables,  we  obtain 


e(o)=/n,  {e"-'.e"'*|  = 
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or 


e(o)=e„(o)eij(o). 


(3.70) 


Thus,  the  characteristic  function  of  a  sum  of  t:r»  independent  random  variables 
is  equal  to  the  product  of  the  characteristic  functions  of  the  items.*  This  rule  for 
calculating  characteristic  functions  can  be  directly  e^anded  for  the  case  of  any 


number  of  mutually  independent  items,  i.e.,  if  Is  the  characteristic  fwciion 


of  the  random  variable  ^  the  characteristic  function  of  the  summation  *7^  = 


K*J 


is  equal  to 


0(0)=  n^uC") 


*-i 


(3.71) 


In  the  special  case  vhen  all  the  items  have  the  same  distribution  (v)  = 

®  l(^). 

0(;>)=e;(o). 


(3.71*) 


The  cited  properties  of  characteristic  functions  make  them  a  particularly  con¬ 
venient  tool  for  the  studsr  of  sums  of  mutually  independent  random  variables.  While 
the  distribution  function  of  a  sum  of  two  inde,;ecdent  random  variables  is  obtained 
by  the  complex  operation  of  convolving  the  di  jtribution  functions  of  the  terms  (cf . 
3.26),  the  characteristic  function  of  this  s«.n  is  found  through  the  si^lc  multipli¬ 
cation  of  the  characteristic  functions  of  Vie  terns.  If  the  characteristic  function 
of.  a  random  variable  is  known,  its  distribution  fhnction  can  always  be  found.  The 
latter  follows  from  the  fact  that,  as  is  evident  from  (3.62),  the  characteristic 
function  0(v)  is  obtained  by  the  direct  transformation  of  the  Fourier  distribution 
function  (J  ^).  Therefore  the  distribution  function  is  obtained  from  the  charac¬ 
teristic  function  by  means  of  the  inverse  Fburier  tran/foraation 


*  Let  us  note  that  in  the  derivation  of  formula  (3*53*)  end,  consequently,  also  of 
(3.70),  it  was  sufficient  to  require  that  ^  ^  and  ^2  ^  uneorrelated. 
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I 


•9 


'%‘P'r 


J’'P'^'*v"’r‘^r-^j^,!T^!t’^-wn^\  •• 


m 

•  (Jt)s=^  J<^?(o)e~'*Vo. 


(3.72) 


Thus,  to  c<tlculate  the  distribution  function  of  a  sum  of  independent  random 
variables  it  is  necessary  first  to  compute  the  characteristic  functions  of  the  items, 
and  then  to  determine  the  inverse  Fourier  transformation  of  the  product  of  these 
characteristic  functions,  uhich  will  be  the  desired  distribution  function. 

6.  Employing  formula  (3.^7),  it  is  not  difficult  to  find  the  expression  for 
the  characteristic  function  of  the  random  variable  ^  =  "^  (^) ,  if  the  distribu.tion 
function  W  (x)  of  the  random  variable  ^  is  known 


e 


(3.73) 


7.  The  method  of  characteristic  functions  can  be  expanded  to  the  aggregate  of 
random  variables  ^2****^n*  characteristic  function  of  the  aggregate  of 
random  variables  ^2'***^n  called  the  mean  value  of  the  random  variable 
S-(‘*^,  ?,+ $„),  where  v^,  V2,..,v^are  real  parameters.  Ifi‘^n(x^,  Xg,.., 
x^)  is  an  n-dimensional  distribution  function  of  an  aggregate  of  random  variables. 


then 


9,  (0|,  Ca. . . .  sj = m,  (e'  •  •  •  +'<•'«> )  = 

__  J...  X 

— 3i— at 

'•times  / 


(3.74) 


Xw*(X|,  xa,...xjdr,dxa...dx^.  ' 

Consequently,  the  n-dimensional  characteristic  function  is  the  n-fold  Fourier 
transformation  of  an  n-dimensional  distribution  function.  Conversely,  the  distribu¬ 
tion  flmction  of  an  aggregate  of  n^  random  variables  is  obtained  from  their  charac¬ 
teristic  function  by  means  of  an  ^fold  inverse  Fourier  transformation. 

On  the  basis  of  the  characteristic  function  6  (>-  of  the  aggregate 

of  random  variables  5^,  $2****^n*  difficult  to  find  the  characteristic 

function  of  any  aggregate  of  k*  <-'n'vsa*iables 
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e*(oi,  o,....o*)=e.(»i.  ‘'2...-t'*.  0....0).  (3.7'^•) 

If  the  random  variables  are  mutually  independent,  their  n-dimen- 

sional  characteristic  function  is  equal  to  the  product  of  the  characteristic  funcJ 
tions  of  each  of  the  random  varialxLes 

'  M 

•*  e, (0,.  0,. . . .  0,)  =  f[  e„  (0,).  (3.75) 

From  (3«7^)  'd^en  v^=y2='...syjj=V  we  obtain  the  expression  for  the  char¬ 
acteristic  function  of  the  sum  of  ^2  '*’'**  "’^^n  random  variables 

,e(o)=0.(».  o.-.  t').  '  (3.76) 

Formula  (3«71)  is  a  special  case  of  (3*76).  v»hen  the  component  sums  are  mutually  in¬ 
dependent. 

8.  Let©  (v  ,  V  )  be  the  characteristic  function*  of  the  random  variables  (^, 
2k  12  i£ 

1|j^)  and  let  the  pairs  of  random  variables  (§2»  ^2^”** ^ 

mutually  independent.  .The  characteristic  function  of  the  random  variables 


is  equal  to 


*-r 

m 

01  (Oi.  ®i)  =  n  ®a*  (''»*  ‘'j)- 


(3.77) 


! 


i  ^ 


If  the  random  variables  and?J^  are  treated  as  components  of  a  plane  vector 
(3*77)  signifies  that  the  characteristic  function  of  a  sum  of  independent  vectors 
(i.e.,  the  resultant  of  the  vectors)  is  equal  to  the  product  of  the  characteristic 
functions  of  the  coaqponent  vectors^  This  rule  remains  true  for  vectors  in  multi¬ 
dimensional  space.  If  all  the  summed-up  vectors  are  subject  to  the  same  two-dimen¬ 
sional  law  of  distribution  ©2(2^12^* 

,  0,(v„p,)  =  {O,(o„o,)r. 

9*  IW-tidirension^  characteristic  functions  may  also  be  engiloyed  for  determin¬ 
ing  the  nixed  moments  of  distribution  of  an  aggregate  of  random  variables  (cf.2.P6), 


*  A  double  index  is  used  with  characteristic  functions  in  the  same  sense  as  with 
distribution  functions  (cf.  reference  on  p.  51). 
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If  there  exists  the  derivative 


^*1  +  *.  +  ...  +  *, 

7.fa,;....j.*-'®*<‘’‘-">--'.)l= 


.*1  +  *«  f  . . .  +  k. 


1 1  ••• 


^  —3D 


»^ . . .  I,)MP  (•  (0,X|+0iJi4- . . .  vt)]  dxidx,...  dx 


i*i  4-*i  +  .*.  +  *• 


(3.78) 


Obviously  (3.65)  is  a  special  case  of  (3.78). 


8,  Computation  of  Distribution  Moments  ? 

'i 

s 

't 

Let  us  apply  the  method  of  characteristic  functions  to  the  calculation  of  dis-  | 

tribution  moments  of  four  types:  normal,  Rayleigh,  uniform,  and  the  distribution  of  | 

,1 

■  I 

the  values  of  a  sinusoid  witt  a  random  phase. 

The  characteristic  function  of  a  random  variable,  normally  distributed  with  the  ] 

J' 

parameters  a,  c-,  is  in  accordance  with  (3.82)  amd  (2.1'’f)  equal  to  } 


I  r  /  ”  2»* 

9(o)  =  -4=-  \e'"e  dx. 


The  substitution  of  ^  ~  ^  complete  the  square  in  the  exponent 

in  the  integrand  function.  Then 

<»-h  _!l 

e(»,=e  '-^^.‘dx. 


Since  with  any  instance  of  a  real 


<•  * 


J  e  dz=y2r.. 
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it  follows  that 
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(3.81) 


at„\ _  I  r  ”*  *  “(s** 


VZn  \  • 


Siqpandlng  (3*81 )  into  a  series  of  povers  of  v,  ve  obtain 


(3.81 •) 


where  (2k) II  is  the  product  of  all  the  even  numbers  of  the  natural  series  to  2k  in¬ 


clusive. 


Comparing  (3*81 •)  with  (3.6?),  we  find  the  distribution  moments  of  the  square 


of  a  normally  distributed  random  variable  to  be 


m.  =  (2a»)*  1^*  "•*)"*! 
‘  '  '  (25Fj« 


(3.82) 


The  characteristic  function  of  a  random  variable,  distributed  according  to  the 
Rayleigh  law  in  accordance  with  (3.62)  and  (2.61)  is  equal  to 


(3.83) 


where  cj3  is  the  Krany  function. 

Bnploying  (3.83)  and  considering  (3.65).  by  successive  differentiation  we  find 


the  Rayleigh  distribution  moments 


m|=s|/^9,  mj  =  2a*,  mj=s3^^^o*, 


or  in  a  general  form 


* 

m,=(2cV  r(-f +  l). 


(3.84) 


Let  us  examine  the  characteristic  function  of  a  random  variable  uniformly  dis- 
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trlbuted  over  the  Interval  of  (a,  b).  Since  the  probability  deisity  within  the 
limits  of  the  indicated  interval  is  constant  and  equal  to  .  —  ,  and  outside  of 


this  interval  is  equal  to  zero.  It  follows  that 


0(0)=  ‘ 

*-aJ  ID  b  —  a  • 


(3.85) 


The  distribution  moments  in  this  case  ore  easiest  of  all  to  find,  if  (3*85)  is  re¬ 
solved  into  the  KacLaurin  series 


(3.86) 


and  the  coefficients  for  like  powers  of  v  are  compared  in  (3*86)  and  (3.87).  As  a 
result  of  such  a  comparison  we  obtain  the  initial  moments  of  uniform  distribution 


_  _  t  6*+i  — a*** 
*+1  b^ 


(3.87) 


From  (3.87) t  as  special  cases,  we  find  the  mean  value  and  the  dispersion  of  a 
random  variable  uniformly  distributed  over  the  interval  (a,  b) 


If  i  *  -il.  then 


- >  +  a  ju  _m  m*— .(AnflL 

“a — rwj— •/Hi—  12  ' • 


”*2* — ^2k  2k  + 


(3.88) 


(3.89) 


Let  us  finally  examine  the  characteristic  function  of  the  values  of  a  random- 
phase  sinusoid.  With  (3.16)  in  mind,  we  find 


I  r 

e  (o)=i 


^ere  JQ(av)  is  a  zero-order  Bessel  function. 

The  corresponding  moments  of  distribution  are  equal  to 


(3.90) 
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Comparing  (3.79)  and  (3.9^)»  ve  conclude  that  the  characteristic  function  of  a 
sum  of  normally  distributed  random  variables  coincides  with  the  characteristic  func~ 
tion  of  a  normally  distributed  random  variable »  but  with  the  mean  value  and  dis> 
persion  determined  ty  formula  (3.93). 

Thus  the  sum  of  any  number  of  independent,  normally  distributed  random  variables 
is  also  distributed  normally,  the  mean  value  of  this  sum  being  equal  to  the  sum  of 
the  mean  values  of  the  terms,  and  the  dispersion  of  the  sum  being  equal  to  the  sum 
of  the  dispersions  of  the  terms.  These  properties  of  mean  values  and  dispersions 
are  special  cases  of  (3*52)  and  (3.58). 

Let  us  apply  the  method  of  charactexdstic  functions  to  the  determination  of  the 
distribution  function  of  the  sum  ^  ^  dependent,  random  variables,  dis¬ 

tributed  according  to  the  normal  law.  For  this,  employing  (3.7^)  and  (2.51).  we 
first  compute  the  two-dimensional  characteristic  function  of  these  random  variables: 


xJI 


^  ,2 


As  a  result  of  computing  the  integral  (cf.  Appendix  V),  we  obtain 

e,(o,,«;J=e  *  “ 


(3.95) 


If  in  (3.95)  it  is  assumed  that  V2  =  0,  then  in  accordance  with  (3.7^')  we  obtain  the 
one-dimensional  characteristic  function  (3*79)  of  a  normally  distributed  random  vari¬ 
able. 

To  determine  the  characteristic  distribution  function  ®  (v)  of  the  sum  of  two 
random  variables,  distributed  according  to  the  normal  law,  it  is  sufficient  now  to 


make  use  of  formula  (3.76): 


0(«')-e,(o.  + 


(3.96) 
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I  J^* 


It  can  be  seen  from  a  comparison  of  (3.96)  with  (3.79)  that  the  obtained  function 

@  (v)  is  a  characteristic  function,  corresponding  to  the  normal  law  of  distribution, 

2  2 

with  a  mean  value  of  a^  +  and  a  dispersion  of  O'  ^  +  2ra^  O'g  +^2  * 

Thus  the  sum  of  two  dependent,  normally  distributed  random  variables  is  also 

normally  distributed,  the  mean  value  of  this  svim  being  equal  to  the  sum  of  the  mean 

2 

values  of  the  terms,  and  the  dispersion  cr  of  the  sum  being  equal  to 

o*s=«j4-2/-a,Oj-f  0*. 

The  obtained  result  is  generalised  for  the  sum  of  an  arbitrary  number  of  de¬ 
pendent  random  variables,  linked  by  an  n-dimensional  normal  distribution  (2.h8).  It 
can  be  shown  that  an  n-dimensional  characteristic  function,  which  corresponds  to  this 
distribution,  has  the  form  of 


*  S  “*'*  ■  T  ^  5]  *1  vm  *k 
oJ=e  *-* 


(3.95*) 


where  r^j^  is  the  coefficient  of  correlation  between  the  random  variables  and  ^  ^ 
*  !)•  Then,  employing  (3*76),  we  find  the  characteristic  function 
of  the  sum  of  n  normally  distributed  random  variables 


e(o)=e,(o,  c....o)=e 


(3.96* ) 


Ftom  (3.96*),  it  follows  that  the  sum  of  the  indicated  random  variables  is  also  dis¬ 
tributed  normally,  the  mean  value  of  the  sum  being  equal  to  the  sum  of  the  mean 
values  of  the  terms,  rtiile  the  dispersion  of  the  sum  is  equal  to 


*"**”*  >t  I 

V  ^  ^ 

Let  us  also  find  the  distribution  function  of  the  summation /V  *  o>a  ®  *  I 

^  I 

lAere  ^  ^ ^  g, , , .  ^  ^  are  Independent,  normally  distributed  random  variables  with  | 

2 

mean  values  of  a  and  dispersion  o-  .  The  characteristic  function  of  each  term  of  | 

this  summation,  l.e.,  of  the  square  of  the  normalized  deviation  $  . ,  is  determined 

-  i 

d 

/* 
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ly  formula  (3.81),  if  it  is  assumed  therein  that  0^=  1.  Therefore  the  characteristic 
function  of  X  ^  is  equal  to 


(3.97) 


idierefrom  by  means  of  an  inverse  Fourier  transformation  we  calculate  the  desired  dis¬ 
tribution  function  of  the  summation: 


1  /* 


0. 


e  .  *>0. 


x<0. 


(3.98) 


Distribution  (3.98)  is  known  as  the  )(  distribution.  Curves  of  this  distribution 
for  several  values  of  n  are  presented  in  Fig.  3.26.  When  n  ^  2,  function  (3.98)  de¬ 
creases  monotonously  for  positive  values  of  the  argument,  and  wljen  n  >  2  it  has  a 
single  maximum  at  the  point  of  x  =  n  -  2,  With  an  increase  in  the  number  of  n,  this 
distribution  curve  approaches  the  curve  of  normal  distribution. 


Fig.  26,  The  X  distribution 

A  special  case  of  (3.98)  for  n  «  1  is  the  distribution  function  (3.1^),  ex¬ 
amined  in  Section  2* 
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10.  Distribution  Function  of  Sum  of  Uniformly  Distributed  Random  Variables* 

Lat  as  find  the  distribution  function  of  two  independent  random  variables,  each 
of  which  is  uniformly  distributed  over  the  interval  of  (a,  b).  Ihe  characteristic 
function  accordance  with  (3»71),  equal  to  the  square  of 

characteristic  function  (3.85),  i.e.,  to 


e„(o)= - 

*  vr(b  —  a)*  ■ 


(3.99) 


From  (3.99)  we  calculate ^  by  means  of  an  inverse  Fourier  transformation,  the 
desired  distribution  function  of  a  sun  of  random  variables; 


f  ,  lao\* 

•  2*  {»  — a)*  J  JS - ® 


(3.100) 


Ebr  substituting  the  variable  £  =  2;  +  a  +  bwe  reduce  (3.100)  to 


— --  f  pnc  *ln*“  j 

{ft — a)  J  6  —  a  '  u* ' 


(3.101) 


The  integral  obtained  is  tabular  and, is  equal  to 

/  I 


I  0, 


It  vay  be  seen  from  (3.101)  that  the  function  even,  therefore  with  a 

negative  argument  its  values  are  calculated  from  the  equality 

•i»(— y)=w,j(y). 

Returning  from  the  variable  jr  to  the  variable  we  obtain  the  following  expression 
for  the  desired  distribution  function  of  a  sun  of  two  independent,  uniformly  dis¬ 
tributed  random  variables 
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«r„(jr)— 


0. 


x<2a, 


ij=^*  2o<x<o4-^. 
(ftLa)!*  o  +  b<x<2b, 


(3.102) 


®I3(0)= 


it»»(6  — 0)*  • 


(3.103) 


Subjecting  (3.103)  to  an  inverse  Fourier  transformation*  similar  to  the  preced¬ 
ing  one,  ve  find  the  desired  distribution  function  of  the  sum  of  random  variables 


«'l3(-')  = 


0. 

i  (x— 3a)* 
i  (b  —  ap  • 

JL  —  3a)J~3fx  —  (b  +  2o)l» 
I  (36-x)t 


2  (ft  — a)»' 


0. 


x<3a, 

3a<x<2a-^b, 

,  2a  +  b<x<a-Jt-2b, 

a-{-2b<x<3b, 

x>3b. 


(3.104) 


Fig.  27  shows  an  initial  uniform  distribution,  and  also  the  distributions  of  a  sum 
of  two  and  a  sum  of  three  independent,  uniformly  '’istributed  random  variables. 


■Irnri 


WCk 

m  %m  i  M  am  ”  • 


9  h 

Fig.  27.  Distribution  of  a  sum  of  Independent,  uniformly  distributed 

random  variables. 
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0,  x>2b. 

This  function  takes  the  form  of  a  triangle,  and  this  distribution  is  therefore  called 
a  triangular  distribution  (sometimes,  a  Simpson  distribution). 

It  is  possible  analogously  to  find  the  distribution  function  of  three  independ¬ 
ent  random  variables,  each  of  idiich  is  uniformly  distributed  over  the  interval  of 
(a,b).  The  characteristic  function  of  this  sum  is  equal  to  the  cube  of  character¬ 
istic  function  (3.85),  i.e.,  to 


with  an  increase  in  the  number  of  n  terms,  the  distribution  function  rapidly 

n(a4b) 

approaches  a  normal  distribution  function  with  the  parameters  of  ^  =  2  • 

jcompare  (3*88^, 

11.  Distribution  Function  of  Randomly  Phased  Sum  of  Harmonic  Vibrations 

Let  us  find  the  distribution  funcUon  of  a  sum  of  independent  harmonic  vi- 
^P^tions  with  constant  amplitudes  and  with  random,  unifomly  distributed  phases: 


{=a,  cos  o)jSt  +  CjCos  ...  4*  0/1  ^0*  “A- 


(3.105) 


The  distribution  function  of  each  of  the  terms  in  sum  (3*  105). was  deteri^ed  in  Sectic 
2  ,  and  the  characteristic  function  corresponding  to  it  is  provided  by  formula  (3.90l| 
This  function  depends  only  on  the  amplitude  aj^,  but  does  not  depend  on  the  frequency  ' 
W.  .  m  accordance  with  (3.71)  the  characteristic  function  ®^(y.)  of  the  sura  ^ 
is  equal  to  the  product  of  the  characteristic  functions  of  the  terms  and  has  the 


form  of 


n 

®fc.(»)=n^(®**')* 


(3.106) 


from  (3.106)  we  calculate,  by  means  of  an  inverse  Fourier  transformation,  the 
distribution  function  *^5^(5)  of  the  randomly  phased  sum  of  independent  harmonic 


vibrations: 


J  dv,  lx|<2Ja*=i1, 


(i.ioeo 


Lncs  i'-’,  (s)=  0  wtien  lx\>A,  this  funcUon  nay,  over  the  interval  of  |  x|<  A,  be 


eaqpanded  into  a  Fourier  series: 
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i^ere 


4  _  OTJt  •  ✓  V 

'.*=5T 


«•  Irtt  m 


•uw=i^  E*'^n4(T)= 

r-— «  *—l 

L  f%  k-t  J 


x\<A. 


(3.107) 


If  the  auplltudes  of  all  the  harmonic  vibrations  are  the  sane  =  a,  A  =  na) , 


\  x\<na. 


(3.107*) 


The  probability,  that  the  absolute  value  of  the  sum  of  the  independent  vibrations  viU 
not  exceed  X.  times  the  amplitude  of  one  term,  would  be 


P I Ili< J«|  =  2 J (x)dx=i  +  j; (2)  tcos ^  dx. 
P||tK»a|  =i  +  l£l  j;(^  ,i„ 


(3.108) 


It  can  be  ^vn  that,  with  an  increase  in  the  number  of  n  terms,  the  distribu¬ 
tion  (3.107)  approaches  the  normal.  Fkom  (2.16)  it  follows  that,  for  a  normal  ran- 

1.  na^ 

dom  variable  n  with  a  dispersion  of  — 5 —  ,  the  probability  is 


P{ln|<2aj  =  2f(l/l)~|. 


(3.108*) 


Table  ^  shows  the  values  of  prcbabilities  computed  according  to  formulas  (3.108) 
and  (3.108*)  for  n  »  10;  it  is  evident  from  this  that  the  randomty-phased  sum  of  10 
harmonic  vibrations  of  the  same  aityHtude  is  sufficiently  close  in  its  statistical 
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characteristics,  to  a  nomal  random  variable. 


Table  4 


X 

t 

» 

S 

4 

.  t 

-  • 

r 

t 

.  according  to(3.to3) 
according  td  (3.108*) 

0,339*5 

0,3473 

0,6221 

0,6319 

0,8168 

0,8130 

0,9267 

0,9281 

r 

0,9766 

0,9756 

0,9944 

0,9931 

0,9991 

0,9985 

The  distribution  function  of  the  randomly  phased  sum  of  two  haraonic  vibrations 


of  equal  amplitude  is  expressed  as  an  elliptical  integral.  From  (3.106),  when  n  =  2 
we  have  0  (v)  =  (av) . 

Presenting  the  square  of  the  Bejosel  function  in  the  form*  of 


r 

^  (av) = ~j  y,  (2ao  sin  <j.)  dij» 


and  employing  (3*106* )•  ve  obtain,  changing  the  order  of  integration  with  respect  to 
V  and  , 

»I2  (*)  =  5  j*  J  4  (20^  sin  •;.)  cos  vxdvd<j>. 

The  inner  integral  is  a  Fourier  transformation  of  a  Bessel  function,  therefore,  con¬ 
sidering  (3.'W),-  we  find  that 

W  /  1 

fyg(2aosin^)cosoxdo=  —  x*  ’  v.2ash»<j», 

_  •  0,  |x|>2asin<|». 


Thus, 


**-  J  r4a*tlttK‘ 


Substituting,  finally,  the  variable  of  integration  u  «  sin  ip  ,  we  obtain  the  desired 
distribution  function  of  the  randomly  phased  sum  of  two  independent  narmonic  vi- 


♦  cf..  Q.  N.  Vatson.  Teoriya  besselevykh  funktsiy  (Theory  of  Bessel  Functions), 
Moskva,  For.  lit.  Pub.  Hse,  1949,  (i.e.,  G.  N.  Watson,  "A  Treatise  on  the  Theoiy 
of  Bessel  Functions",  2nd  Bd.,  N.  I..  1944). 
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brttlons  in  the  fom*  of 


=  |  f  - — 1/^1  -  ii) 


Wj,(x)=Owheni  x  \  '>2a. 


»hen  |x|<2o, 


(3.109) 


vhere  k  is  a  oouqplete  elliptical  integral  of  the  first  kind. 


Fig.  28,  Distribution  of  a  randomly  phased  sum  of  independent  vibrations. 

Fig.  28  shows  the  initial  randomly  phased  distribution  of  harmonic  vibration, 
and  also  the  sums  of  two  and  three  independent  vibrations.  With  an  Increase  in  the 
number  of  n  terms,  the  distribution  function  of  a  siun  rapidly  approaches  the  normal. 

12.  Generating  Function  of  Discrete  Random  Variable 

For  a  discrete  random  variable  -.diich  takes  integer  values  of  i  with  probabili¬ 
ties  of  Pp,  a  role  analogous  to  that  of  the  characteristic  function  of  a  continuous 


random  variable  is  played  bsr  the  generating  function,  defined  ty  the  equality 


(3.110) 


Qy  means  of  the  generating  function  it  is  easy  to  calculate  moments  of  distribution, 
as  well  as  dlstritutions  of  sums  of  discrete  random  variables.  8y  successive  dif- 


•Let  us  note  that  distribution  ^^^(x)  in  the  form  of  integral  (3.109)  is  obtained 
directly  from  (3.24)  as  a  convoluiion  of  the  two  distribution  functions  0.16). 
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ferentiation  ve  obtain 


irtierefron 


e"(o)  =  £r(r-l)»'"p,, 

r«*0 

o» 

e'(l)  =  5]rp,=m,. 

r-0 

Oi 

=  etc. 


(3.111) 


(3.112) 


It  is  possible  to  show  that  the  generating  function  of  a  sun  of  independent 
random  variables  is  equal  to  the  product  of  the  generating  fimctlons  of  the  terms. 

For  a  discrete  random  variable  distributed  according  to  Poisson's  law  the  gener¬ 


ating  function  is 


(3.113) 


The  generating  function  of  a  sum  of  two  random  variables,  distributed  according 
to  Poisson's  law  with  parameters  of  X  ,  and  ^  -  is  equal  to 


e  (o)  =  01  (s)  0^  (e) = e^' 


(3.11*^) 


i.e.,  is  equal  to  the  generating  function  of  a  Poisson  distribution  with  a  parameter 
of  +  X  2  • 

Thus  the  Poisson  distribution,  just  as  the  normal,  reproduces  itself  in  the  ad¬ 
dition  of  independent  random  variables. 

Prom  (3.113)  (3.111)  «id  (3.112)  it  is  not  difficult  to  find  the  mean  value  and 
dispersion  of  the  Poisson  distribution 


mi=e'(I)  =  l, 

Afj = m  j  —  m* = 0"  ( I ) -f  m ,  —  m| = X* 1  _  1* = 1. 
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Chapter  IV 


LIMITING  THEOREMS 


1»  The  Lyapunov  Theorem 

One  of  the  in{)ortant  results  of  Section  9  of  the  preceding  chapter  was  the  fol¬ 
lowing:  the  sum  of  independent,  normally  distributed  random  variables  is  also  noj>* 
mally  distributed.  On  the  other  hand,  the  distribution  curve  of  a  sum  of  three  uni¬ 
formly  distributed  random  variables,  shoYfn  in  K.g,  27,  approaches  the  normal  distri¬ 
bution  curve. 

The  same  phenomenon  has  also  been  noted  with  respect  to  the  distribution  of  a 
randomly  phased  sum  of  harmonic  vibrations,  as  well  as  to  the  distribution 
(Figs.  26  and  28). 

Further,  in  a  succession  of  independent  tests,  the  number  of  k  occurrences  of 


an  event  in  n  tests  may  be  regarded  as  the  sximmation  of  ^  y  independent  dis- 


Crete  random  variables,  each  of  which  can  take  only  two  values;  unity  with  a  proba¬ 
bility  of  p,  and  *ero  with  a  probability  of  9  *  1  -  P*  According  to  Liq^ace^s 
iasraptotic,  formula  given  a  large  valu.es  of  n  the  distribution  of  the  number  of  ocoir- 
ences  of  events,  i.e.,  the  distribution  of  a  sum  of  independent,  discrete  random 
variables  of  an  Indicated  type,  may  with  some  accuracy  be  considered  as  normal. 

The  question  arises:  cannot  this  property  of  a  sum  of  individual  random  vari¬ 
ables,  proved  in  several  special  cases,  be  generalized  to  become  an  arbitrary  law  of 
distribution  of  the  terms.  The  answer  to  this  is  affirmative,  and  constitutes  the 
contents  of  the  so-called  central  limiting  theorem  of  probability  theory,  proved 
first  by  lyapunov  and  subsequently  refined  by  other  mathematicians..  The  great  im¬ 
portance  which  this  theorem  has,  in  natural  science  and  in  engineering  (including 
radio  engineering),  is  linked  with  the  fact  that  in  practice  it  is  very  often  nec¬ 
essary  to  investigate  phenomena  idiich  are  affected  by  a  large  number  of  independ¬ 
ently  acting  random  factors. 
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As  has  been  pointed  out  in  the  preceding  chapter,  the  most  convenient  and 
simple  method  of  studying  a  sum  of  Independent  random  variables  is  the  method  of 
characteristic  functions.  This  method  will  be  used  below  for  proving  the  lyapunov 
theorem  with  respect  to  suns  of  independent,  continuous  random  variables.  It  may 


also  be  employed  for  discrete  random  variables. 


Let  us  then  examine  the  summation  of  ^  ^  ^  ^  independent  random  variables. 


r=i 


the  distribution  functions  of  idiase  individual  terms  are  the  same  and  equal  toCi>(x). 


Then  the  mean  value,  a,  and  the  dispersion.  O'  %  of  each  term,  are  equal  to 


OB  *ae 

a=  J  xw{x)dx,  J  (x  —  a)-w{x)dx. 


and  the  mean  value  and  dispersion  of  the  summation,  ^  ,  are,  in  accordance  with 
(3.52)  and  (3*58)  equal  to 

A/jj;|=n32. 


('f.D 


We  shall  be  seeking  the  distribution  function  of  the  normalised  deviation  of  the  sum, 
i.e.,  of  the  random  variable 


9Yn  Kn  * 

r»l 


(4.2) 


For  this  it  is  useful  first  to  find  the  characteristic  function  of  each  term  of  the 
sum  (4.2),  and  then  to  employ  the  rule  already  known  to  us,  namely  that  the  charac* 
teristic  function  of  a  sum  of  independent  random  variables  is  equal  to  the  product 
of  the  characteristic  functions  of  the  terms. 

Since  for  any  tera,  the  distribution  function  is  the  same  and  equal  to  u)(x), 
the  characteristic  function  of  the  deviations  4(r  =  1,  2,...n)  will  be  the  same 
and  equal  to 

e(.)=  f 

(».3) 
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!  » 


I  'i 


I  ^ 


Expanding  e— into  an  exponential  series,  and  changing  the  order  of  summation 


and  integration,  we  find 


«(„)=? 

Jao  *-0 

=  |j-*y  |(x-a)*a»(x)dx. 


(4.4) 


where  are  central  momer/s  of  the  distribution  U)  (x).  Expansion  (4.4)  is  obtained 
from  (3.67)  through  a  replacement  of  mj^  by  I^,  since  here  0 (v)  is  the  characteristic 
function  of  the  normalized  deviation  of  a  random  variable.)  The  characteristic 

<5  k-  " 

function  of  the  normalized  deviation  is,  according  to  (3.64),  equal  to 


Qf  »  » 

*-o 


(4.5) 


The  desired  characteristic  function  of  the  sum  7^  is,  according  to  (3.71*),  equal  to 


e»<”'=i®(rh)f 


(4.6) 


Substituting  into  (4,6)  the  expression  0  (  (4.5),  we  obtain 

(Ml  =  0,  Mg* 

e  ri  — — 

W  j  in  fi^n^a  +  •  *  * j  *  (4^7) 


The  approach  to  the  limit  of  n--*-oo  in  (4,7),  results  in  an  indeterminate  form  of  the 
type  1^.  To  clear  up  this  indeterminacy,  we  examine  •  Then  from  (4.7) 


it  follows  that 


t  \  t  ft  ®*  I  MjO*  I  \ 

In  0,^  (0) — /I  In  ^  •  7 


(4.8) 
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Expanding  the  logarithm  into  a  series,  we  obtain 

In  («) = - T  -  6*5^ «^r + •  •  • 

Approaching  the  limit  when  n — »-oo,  ve  find  from  (4,9)  that 


limln0,»=-?. 


lAerefrom 


2 


(4.9) 


(4.10) 


From  (4^10)  it  follows  that  the  distribution  function  Wj^^(x)  for  the  normalized  devi. 
ation  of  a  sum  of  independent  random  variables  tends  toward  the  norm^  as  the  number 
of  terms  is  increased 


lim  w^Jx)  =  w^x)  = 


(4.11) 


or 


(4.12) 


Bnploying  the  egression  (4.1 )  of  the  mean  value  and  dispersion  of  a  sum,  it  is  not 
difficult  to  pass  from  the  distribution  function  4)  ]^(z) .  of  a  normalized  deviation, 
to  the  distribution  function  of  a  sum  of  independent  random  variables 

'  K2K/t» 

In  this  manner  it  has  been  proven  that  the  law  of  distribution  of  a  sum  of  in¬ 
dependent  randan  variables  having  the  same  distribution  function  tends,  as  the  sum 
of  the  terns  is  Increased,  toward  the  normal,  regardless  of  idiat  the  distribution  of 
the  terms  had  been. 

With  certain  supplenentaiy  assumptions,  the  lyapunov  theory  is  valid  also  in 
the  case  when  the  distribution  functions  of  the  terms  are  unequal.  It  nay  be  ex¬ 
panded  even  to  certain  cases  when  the  terms  are  not  independent  [5-]  . 

On  the  other  hand,  the  distribution  of  a  sun  of  independent  random  variables 
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does  not  always  converge  towards  the  normal.  In  tlie  derivation  presented  above  it 
was  assumed  that  the  mean  values  and  dispersions  of  the  terms  exist  and  are  finite. 
However,  this  condition  might  not  be  met  even  in  practice.  Suppose,  for  instance, 
there  is  studied  the  distribution  of  a  sum  of  random  variables,  each  of  which  is 
equal  to  •  ,  with^P  being  distributed  according  to  the  Rayleigh  law 

(2.61)*.  Then  the  distribution  function  of  the  random  variable  7^^  is,  in  accordance 
with  (3.7)*  equal  to 

A  mean  value  and  a  distribution  for  each  term  (i.e.,  for  the  random  variable  ) 
do  not  exist,  since  the  integrals 


m,= 


J  7  c  J  ® 


diverge. 

The  general  conditions  (necessaryand.suff icient) ,  for  the  convergence  of  the 
distribution  function  of  sums  of  independent  random  variables  towards  the  normal  law, 
have  been  found  in  recent  years  [5]»  Parallel  to  this  there  arose  the  question  as 
t-.  ..lat  law,  besides  the  normal,  can  be  limiting  for  sums  of  independent  random  vari> 
ables.  It  turned  out  that  the  limiting  laws  are  far  from  exhausted  fay  the  normal 
law.  A  large  class  of  normalized  sums  of  independent  random  variables  (including 
those  for  which  terminal  numerical  characteristics  do  not  exist)  have  distributions 
tending  toward  the  so-called  stable  laws.  Each  stable  law  may  be  reduced  to  a 
canonical  form,  determined  by  the  characteristic  function 

e(«)=e  I  ^ 


e(a)=e  I  «  =  1, 


*  Sums  of  this  type  occur  in  research  on  the  influence  of  propagation  conditions 
(fading)  in  radio-relay  communication  systems  (cf,  also, Section  5  of  tbis  chapter), 
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When  o<  9  2,  the  stable  law  corresponds  to  the  normal  law. 


2, _ Estimate  of  the  Rate  of  Convergence  Towards  the  Normal  Law. 


For  practical  applications  of  the  Ijrapunov  theorem  it  is  necessary  to  estimate 


the  asymptotic  equalities  (4.12)  and  (4.13)  In  relation  to  the  number  of  n  terms  and 


to  the  form  of  the  distribution  function  Oi  (x). 


A  correction  to  the  normal  law  for  the  distribution  function  of  a  sum  of  inde¬ 


pendent  random  variables  is  obtained  fx^>m  an  examination  of  expression  (4.8).  The 


function  l5®]ji(x)  exponential  series  in  terms  of  v,  the  coefficients  of  idiich 


depend  on  ^  and  on  the  central  moments  of  the  initial  distribution  of  the  terms. 


Depending  on  the  precision  required  for  the  evaluation  of  the  approach  of^^j^(30  ^ 


the  normal  law,  it  is  possible  to  restrict  one*s  self  to  the  necessary  number  of  the 


terms  in  this  series.  If,  for  example,  it  is  required  to  retain  terms  of  an  order 


not  exceeding  -  ,  then  from  (4.9)  we  find 


a  series,  we  obtain  with  a  precision  of  tlie  order  of  0(n'’^' *) 


(»)=«■  ■  ('  +  *,"■  -JtSj  •') 


(4.14) 


from  (4.14)  we  find  with  the  same  precision  the  distribution  function  of  the 


normalized  deviation  (4.2) 


Introducing  the  coefficients  of  a^ymmetiy  and  excess  of  the  initial  distribution  n(x) 
according  to  formulas  (2.82)  and  (2.84)  and  expanding  the  exponential  function  into 


0» 


e  *  do. 


(4.15) 


the  right  side  of  (4.15)  there  stands  the  sum  of  four  Integrals,  the  first 
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of  ^diich,  according  to  (3.79),  is  equal  to 


«Q 


^  -!•*-£!  ,  _£! 
''rfo=:_L_e  * 
KS  • 


(4.16) 


The  remaining  tw>  are  obtained  from  (4.1 6)  ty  differentiating  with  respect  to  x: 


=  — —  ^e“^)-  H  f  \  » 


dv  = 


2 


(4.17) 


d*  [  I 

‘I' 


=i433('"^)=".Wj4«'^. 

d*  (  1  f  .  \  (  i)»r  . 


(4.18) 


dv=s 


(4.18*) 


where  52(5),  Hj^(x),  Hg(x)  arc  Hermite  polynomials  (cf.  Appendix  /II). 

Substituting  (4.16),  (4.1?),  (4.18)  and  (4.18*)  into  (4.15),  we  find 

ji* 

»u(*)=j^e  *  [l~^/^3(>f)+2iH//«(x)  +  7^//6(jf)].  (4.19) 

Prom  (4,19)  it  follows  that  the  distribution  of  the  normalised  deviation  of  a  sum  of 
random  variables,  iMch  have  synmetrical  distribution  functions  of  (k  *  0),  converges 
upon  the  normal  distribution  more  rapidly  than  the  normalised  deviations  of  a  sum  of 
random  variables  for  idiich  the  distribution  functions  are  asymmetrical  (k:^  0). 

Prom  (4.19)  we  find  also  the  expression  of  the  distribution  function  of  the  sum  of 
independent  random  variables  itself 
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“4 


<«-<»«)» 

''■•('>=ii;‘  “■ 


+^«‘r;??)+^''.(T7f)J- 


(4.20) 


;3»  Examples, 

As  examples  Illustrating  the  application  of  the  formula  obtained  above,  let  us 
find  and  compare  the  distribution  functions  of  the  normalized  deviation  of  a  sum  of 
seven  random  variables  for  three  special  cases:  a)  the  terms  are  distributed  accord¬ 
ing  to  the  Rayleigh  lav,  b)  the  terms  are  distributed  uniformly,  vith  a  zero  mean 
value,  c)  the  terms  are  distributed  according  to  the  lav  of  (3*16)  (i.e.,  a  sum  of 
sinusoids  vith  random  phases  is  examined  X  Mb  assume  that  the  dispersions  of  the 
terms  are  the  same  in  all  three  cases. 

Table  5  shovs  numerical  characteristics  (mean  values,  dispersion,  coefficients 
of  asymmetry  and  excess)  of  random  variables  for  the  indicated  distributions,  as  veil 
as  for  normal  distribution. 

Baploying  (4,19),  ve  find  the  distribution  functions  of  the  normalized  devi¬ 
ations  of  the  sums  of  random  variables  in  the  indicated  special  cases 

— 5l 

«'.(^)  (0.995-  0,I23x+  0,0l06x»+0,041x3-0.00I8x«) 

(Rayleigh  distribution  of  the  terms), 

(0,979+  0,0W  -  0,007x*) 


y 


§ 


(uniform  distribution  of  the  terms), 

_£» 

W ^  + 0.054X*  -  0,009x«) 

(sum  of  sinusoids  vith  random  phases). 
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Table  5 


Numerical  Characteristic 

Distribution 

Normal 

Rayleigh 

Uniform 

Sinusoid 

Mean  value 

0 

/E 

0 

f 

0 

Dispersion 

2 

2 

2 

2 

Coefficient  of 
Asymmetry 

0 

i 

0,63 

0 

0 

Coefficient  of 
excess 

0 

-0.3 

-1,2 

•  < 

->.5 

Fig.  29*  Convergence  of  sums  of  Independent  random  variables  to  the  normal. 

The  figure  shows  the  differences  between  distribution  functions  of  the  normalized 
deviations  of 

1)  the  sum  of  seven  Rayleigh  random  variables  md  the  normal, 

2)  the  sun  of  seven  uniformly  distrituted  random  variables  and  the  normal, 

3)  the  sum  of  seven  vlbrntions  with  random  phases  and  the  normal. 
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In  Fig.  29  is  plotted  the  differences  between  the  distribution  functions  Wj^(x), 
and  the  limiting  function  ‘*>^(5)  of  normal  distribution.  From  the 
figure  it  is  evident  that  the  distributions  of  a  sum  of  sinusoids  with  a  random  phases 
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where  a  and  ^  are  the  mean  valuest  while  O' ^  and  O'  ^  are  the  dispersions  of  each  of 
the  random  warlables  4fOnd^^ respectively. 

The  tWDodimenslonal  characteristic  function  of  either  of  the  vector."  ( ® » 
^  —  b  )  is  equal  to 


m 


(4.23) 


Bcpandlng  the  exponent  in  the  Integrand  expression  into  an  exponential  series*  and 
changing  the  ot^er  of  sunnation  and  integration*  we  ^nd 


D  D "STT  ®2  J  J  (*— a)*(y  —  y)dxdy. 


(4.24) 


km4t  rmk 


(4.25) 


where  denotes  nixed  moments  of  the  (k  +  r)-th  order  of  the  random  variables 
^  j  and  ^ 2*...n)*  idiioh  do  not  depend  on  J*  since  any  pair  of  these  ran¬ 
dom  variables  has  tjie  sane  distribution  function  u) 

The  diaracteristic  function  for  a  normalised  vector  is  equal  to 


•< 

I  4 


i*+r  Ali*+0 


*+r  ®l  ^2  • 


(4.26) 


The  desired  characteristic  function  of  the  sum  of  the  normalized  vectors  (4.22) 
is*  according  to  (3.77*)*  equal  to  the  n-th  power  of  the  characteristic  function 
(4.26)*  i.e.*  •  - 


!•«  "li*” 

\LLt~M7i - FTr*'**'* 


k^fmO 


p*  r/  _r,  «^i+2rir,a|  +  o| 

J  ^  ~ 


_ 

V~ 
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I  i 


-  I 


A 


•nd  of  A  sum  of  unifomly  distributed  terms,  converge  towards  the  normal  law  more 
rapidly  than  a  sun  of  random  variables  with  a  Rayleigh  law  of  distribution,  although 
at  first  glance  it  seems  as  if  the  Rayleigh  distribution  were  closer  to  the  normal 
than  the  uniform  distribution  or  that  of  (3»l6).  This  is  because  the  Rayleigh  dis¬ 
tribution  is  asymmetric,  while  the  other  two  are  synnetrical  distributions. 


4.  Two-Dimensional  Central  limiting  Theorem. 

The  central  liniUng  theorem  of  probability  theory  can  also  be  expanded  to 


multidimensional  cases. 

As  has  been  indicated  above,  an  aggregate  of  random  variables  ^ 


(4.21) 

For  the  solution  of  the  problem  at  band  there  will,  as  in  the  one-dimensional 
case,  be  enqployed  the  method  of  charactenstic  functions,  there  being  first  also  de¬ 
termined  the  distribution  funcUon  of  the  sum  of  normalized  vectors  with  coordinates 

Of  i 

M  «,  r  n  « 
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may  be  treated  as  conponents  of  a  vector  in  an  n-dinensional  space. 

¥e  shall  examine  the  sun  of  the  mitually  independent  vec'bors  Iiaving  identical 
n-dimcnsional  distribution  functions  wv^  •••  their  conponents.  Then, 

with  certain  limiting  distributions,  the  component  of  the  resultant  vector  (i,  e,, 
the  sum  of  the  vectors)  approaches  the  n-dimensional  normal  law  of  distribution. 

Ws  shall  examine  in  detail  the  two-dimensional  case  of  this  theorem,  which 
figures  in  many  applications,  with  ihe  ^\xffose  of  providing  at  the  same  time  a  cor¬ 
rection  to  the  normal  law  for  the  distribution  function  of  a  sum  of  mutually  inde¬ 
pendent  vectors  in  a  plane,  in  relation  to  tlie  number  of  n  sumned-up  vectors  and 
their  numerical  characteristics. 

I^t  ),  {^2*  ^  ^  n’  ^n^  ^  components  of  independent 

vectors  in  a  plane,  subject  to  the  same  two-dimensional  law  of  distribution 
Vfe  find  the  distribution  function  of  the  resultant  of  these  vectors,  the  components 
(^  »^)  of  idiich  are  equal  to 


1 


where  r  is  the  coefficient  of  correlation  between  the  random  variables  ^  ^  and 
and  k2  are  the  coefficients  of  asymmetry  of  each  of  the  variables  ^  j  and 


respectively.  Symbols  k^2  ^^21  ^^•2?)  denote  the  foU.owing  integrals: 


m  m 

y)dxd>f. 

»  3.^ ..00 

«0  OD 

I  I  y)dxdy. 


(4.28) 


(4.29) 


To  investigate  the  behavior  of  the  function  ®  2n^^r  ^2^  large  values  of  n, 
we  examine 

6  + - 2 - 


Bqpanding  the  logarithm  into  a  series,  we  obtain 

/_  "i+Zrv.Vi+pf 

•0  “*.(*'!»  ^2) - 2 - -  — 

I  /’Vi+ks®*  ,  ^  , 

""yrV  «  ^  a 


(4.30) 


is  the  number, n.  of  the  terms  increases  without  limit,  we  have 

•  .jt _  .9 


('*.31) 


idiich  coincides  with  the  characteristic  function  of  the  normalized  deviations  of  two 
random  variables,  subject  to  the  two-dimensional  normal  law  of  distribution  (  comp. 

3.95). 

In  this  manner  it  has  been  proved  that  the  probability  distribution  of  the  com¬ 
ponents  of  a  sum  of  normalized,  mutually  independent  vectors,  subject  to  the  same 
law  of  distribution,  tends  towards  the  two-dimensional,  normal  distribution,  as  the 
number  of  terms  is  Increased,  regardless  of  what  had  been  the  distribution  of  the 
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components  of  the  sdded  vectors* 

Having  an  expression  of  the  tvo-dlmenslonal  distribution  function  of  the  com¬ 
ponents  of  a  normalised  resultant  vector 


y) 


x*—Jrxy'i-yt 


Z<l-r«) 


2* /I- 


(4.32) 


it  Is  not  difficult  to  find  the  asymptotic  expression  for  the  distribution  function 
of  a  non-noraallzed  resultant  vector 

rrzTi  ^ 

(4.32*) 

_ n*— (Ml-  »iJt— no)  (y— /i»)  ,  (y— iia)i  ) 

teMwIi  :  9  ““  +  I*  I 

Xe 


y) 


2isn«i«j  /"  1  — r* 

_  I  I  (»■— no)*  2r(x— no)  (y— nfr)  .  (y— ni)*  ) 

*?  ”T"1. 

Baploying  (4*30)  it  is  possible,  with  a  margin  of  error  on  the  order  of  l/n,  to 
establish  a  correction  for  the  two-dimensional  normal  law  of  distribution  (4*32). 
Since  from  (4*30)  it  follows,  within  the  indicated  margin  of  error,  that 


a  /  %  -  4- 


*uu,f|+ 

2 


I  +  kjt-2 


■f" 


therefore,  subjecting  (v^,  Vg)  to  an  inverse,  two-dimensional  Fourier  trans- 


foraation,  we  find  the  following  expression  for  the  distribution  function  O)  «  (x,y) 

2n 


of  the  components  of  the  rc.*niltant  vector  ( ^  ) 

X  [■  -pr  t  )] 

Breaking  down  the  integral  of  the  sum  into  a  sum  of  integrals,  and  employing  re¬ 


lationships  analogous  to  (4,1 6)- (4. 18),  we  find 

•  -  2«-f»)  ^ 


x*—2rxy-\-y> 

I'  OKJ 


I 


X  {(k,  -  r*„)  ((ry  _ x)’ -  3 (I  -  r*) (ry  -  x)]  + 

(k,  —  r/tji)  [(rx  -  y)»  -  3  (1  -  r*)  (rx  —  y)l  +  3ft, jy  (1  -  r*)  X 
l(ry-x)»->(l-/'*)j4-3ft„x(l-r*)[(rx-y)2~(I-r*)l)l. 


(4*33) 
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i.  The  "Random  Walk"  Problem 


The  two-dimensional,  central  limiting  theorem  of  probability  theory,  proved  in 
the  preceding  section,  is  closely  linked  to  the  famous  problem  of  the  random  walk, 
encountered  in  many  applications  (reflection  of  radar  signal  from  clouds,  multi beam 
propagation  in  radio-relay  comauni cation  (fading),  Brownian  movement,  direction  of 
plant  growth,  etc.). 


Pig.  yi,  Ratxlom  migration  of  a  point  in  a  plane. 

Let  us  examine  a  point  which  can  move  about  in  a  plane,  consecutively  travers¬ 
ing  linear  segments,  as  shown  in  Fig.  30*  The  lengths  of  these  segments  and  the 
directions  of  the  moves  are  random,  and  are  subject  to  a  certain  two-dimensional 
distribution  function.  The  problem  of  the  random  walk  consists  of  determining  the 

probability  density  W_  (x,y)  of  the  fact  that,  starting  from  the  origin  of  the  co- 

■“Zn  ^ 

ordinates,  the  point  after  n  consecutive,  mutually  independent  moves  will  have  the 
coordinates  (x,y).  It  is  obvious  that  any  move  may  be  treated  as  a  vector  with  the 
random  components  (<^  ^  j^);  then  the  position  of  the  point  after  n  moves  will  be 

indicated  by  the  end  of  the  resultant  vector  ( <^  ,  ) ,  the  components  of  which  are 

equal  to  the  summations  (4.21). 

For  physical  applications  it  is  useful  to  regard  the  indicated  rest  ltant  of  the 
vec  iors  as  the  sum  of  n.  isoperiodic  vibrations  with  a  given  distribution  function 

amplitudes  f  ^  ^  J  and  phases  =a»-c+<j  (k  =  1,  2,...n). 

In  such  a  form  the  problem  of  random  moves  is  a  generalization  of  the  problem  examliied 
■Jbi  of  Ch.  3*  where  it  was  assroed  that  the  amplitudes  of  the  vibrations 
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were  constant,  and  that  the  phases  were  uniformly  distributed  between  0  and  2  , 

One  of  the.  sources  of  interference  in  the  reception  of  a  radar  signal  is  re¬ 
flections  from  such  objects  as  clouds, S'jnetallic ’.strips,  vegetation,  etc.  The  mathe¬ 
matical  description  of  the  reflections  of  a  radar  signal  from  these  objects  is  based 
on  the  assumption  that  they  are  caused  liy  reflections  from  a  large  number  of  inde¬ 
pendent  reflectors.  A  problem  Important  in  its  applications  lies  in  determining  the 
distribution  function  of  the  amplitudes  of  the  resultant  signal,  obtained  from  the 
addition  of  all  these  reflected  vibrations.  If  it  is  assumed  that  the  amplitude  dis¬ 
tribution  of  the  reflected  signals  depends  only  on  tne  distance  r  to  the  reflector 
and  is  equal  to  (r)  for  any  of  the  signals,  and  the  phases  of  these  high-frequency 
vibrations  are  equiprobable  and  do  not  depend  on  the  amplitudes,  then  the  problem  at 
hand  may  be  solved  exactly*. 

Identifying  each  of  the  reflected  signals  with  a  vector,  the  length  of  which  is 
=  ^  ,  and  whose  direction  is  equiprobable,  we  return  to  the  problem 

of  the  preceding  paragraph, 

Baploylng  (3*29) »  we  find  the  two-dimensional  distribution  function  of  the 
amplitude  and  phase  of  any  of  the  reflected  signals  (vectors  with  components  of  ^ 

.  .  ..  _ 


V't  {r,  f ) = ^  O'  (0  =  '•“'2  y)- 


(^.34) 


The  two-dimensional  characteristic  function  of  any  of  the  vectors  ( 4^  ^  j^) 


is  equal  to 


J  w^{x,  y)e‘' (/xdy  = 

— 0» 

C>  in 

Integration  on  U> ,  as  in  $e^ion  5.of  Ch*  leads  to  a  zero~order  Bessel  function, 

/  OB 

(4.35) 


(4.35) 


*  This  assumption  is  valid,  if  the  distance  from  the  radar  receiver  to  the  group  of 
reflection  sources  is  much  greater  than  the  distances  between  the  sources.  Moreover, 
the  present  problem  does  not  consider  the  directivity  pattern  of  the  antenna. 
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Ihe  characteristic  function  of  the  resultant  vector  is»  in  accordance  with  (3*77* )t 
equal  to  the  n-th  power  of  characteristic  function  (4,35); 


00  00  00 

= J  J  •  •  •  1“' •  O' (O ^  (r,  X 


(4,36) 


•  tines 


X  Jo  ('■a  “'Joiry  v\-\-v\]  dr,  dr, . . .  dr,. 


From  (4,36)t  hy  means  of  a  tw -dimensional  inverse  Fourier  transformation  we 

find  the  desired  distribution  function  of  the  components  of  the  resultant  vector 

00  00 

•’»(«.  y)=ir.I  J  %M.  Jc.,= 

•“00  —00 

•  *  00  00 

0  0  .^—00 
Twines 

•  ♦  •  dvldv^r^ . . .  dr^. 

Etsr  a  substitution  of  the  integration  variables  v^  =  s  cos  .  Vg  *  £  sin  i|>  ,  we  re¬ 
duce  the  expression  ^  2n^- *2^  ^ 


00  00  ooa* 

»j<,(jf.  y) =47* J  JJ-''o('’>^)- 

■  Alpines 

...  do  (r,s)  e-  sdsd^dr,  ...dr. 


(4,37) 


If  in  the  distribution  function  «  ^{x,^  we  now  pass  to  the  variables  r  and  ,  then 
in  accordance  with  (3<29)  we  find  that 

00  00 

?)=  rwj  Jr  cos  f,r  sin  9) = j  ...jw(r,)... 

00  a<  '^^ISros 

Xe-'''*<”<^>sd<}.d5dr,...dr,. 

Integration  with  respect  to  (jj  yields  a  zero-order  Bessel  function,  then 
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(4.38) 


o»  m 

?)=2rJ •••] 

"+»“?dines 
. . .  (»’«*)  •/()  (rs)  sdsdr^...  dr^, 

from  whi<*  we  obtain  the  distribution  function  of  the  amplitude  of  the  resultant 


signal 


2c  eo  oe 

aft  times 

...to  (r\  Jq  {r,5) . . .  /o  (V)  /o  (r,  s)sdsdri . . .  dr^. 


(4.39) 


Considering  that  th<  variables  r^,  in  integral  (4.39)  are  separable, 


we  find 


00  /OO 

'^in(r)==rj(ju; 

U  \u 


(o)  Jo  1  r  >  0. 


(4.40) 


The  corresponcjing  integral  distribution  function  i.e.,  the  probability  that 

the  amplitude  of  the  resultant  vector  will  not  exceed  R,  is  equal  to 


/too  /» 


w  (»)  /q  (w)  dv  I  rsJg  (rs)  dsdr. 


Since 


a 

I  rsJo  {rs)  dr  =  /?/,  (/?s), 

^  /OO  \  n 


(4.41) 


If  the  length  of  the  sunmed-up  vectors  is  constant  and  equal  to  a,  then  U)  (r) 
*  S  (r  -  a)  and  from  (4,41 )  we  find 


00 


(4.42) 


The  probability  that  the  length  of  the  resultant  of  the  vectors  (the  length  of 
each  is  equal  to  a,  and  its  direction  is  random)  will  not  exceed  the  length  of  one 
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r'  ■ 


*  i 

i 

’  i 

i 

<■ 

jS 


tern  ist  in  this  case,  equal  to 

f  “ 

^  00 

a  resultant  obtained  by  KLlver  (i.e..  Cleaver?)  [?]  . 

It  is  of  interest  to  examine  a  case  where  the  number  of  n  reflected  signals  is 
great,  which  actually  occurs  in  practice  (for  instance,  the  number  of  reflected 
vibrations  from  drops  of  rain).  On  the  basis  of  the  two-dimensional  limiting  theorem 
it  is  possible  to  say  in  advance  that  the  distribution  of  the  vector  components  of 
the  resultant  signal  (after  an  appropriate  normalization  of  the  terms)  will  be  ap¬ 
proximately  normal.  Let  us  find,  employing  (h-.i|C),  a  correction  to  tids  normal  law 
in  relation  to  the  number  n  and  to  the  distribution  moments  W(r)  of  the  amplitudes 
of  the  reflected  vibrations. 

The  distribution  function  of  the  amplitude  of  the  normalized  resultant  vibration 
is  obtained  from  (4,40)  by  the  substitution  of  P  r  r* 

I  I  »n  n  "■ 


(P)  =  K  ^  P  J  y  ^  {vs)  do  j  s/,  (sp 


X 


(4.^3) 


Xds 


zjQ{pz)dz. 


Expanding  the  Bessel  function  into  an  exponential  series 

(-i)*p**a2* 


■(«K4r)=S^ 
*-0  * 


and  changing  the  order  of  summation  and  integration,  we  obtain 


nhere  are  the  even-order  Initial  laoments  of  the  distribution  function  ^  (r)  of 
the  amplitudes  of  the  reflected  signals. 


Let  us  write  L(*)  =  ( I  —  ,  then 

*  #4  ft  .  .  ^  ^ 

"  14  n 


/  .  mtz*  z*  ,  \ _ 

V 2/1"^  I6m*rt»  ani>~'--J  — 


idierefrom 


1(2) -^e 


Substituting  the  obtained  expansion  of  the  Integrand  function  into  (4.43),  we 


obtain  with  a  precision  of  terns  of  the  order  of  (cf.  Appendix  VI) 

n* 


m  f* 


and  since 


.F,{-2.1.'4)=(1-p’+^). 


(4.144) 


From  (4.44)  we  find  the  desired  distribution  function  of  the  amplitude  of  the 


resultant  signal,  idien  the  number  of  h  reflections  is  large: 


«^uM=;gr''  [•+4(^-2)(4?  -$+•)] 


(4.45) 


Thus,  with  a  large  number  of  reflections,  the  amplitude  of  the  resultant  signal 
is  distributed  according  to  the  Rayleigh  law.  This  result  corresponds  fully  to  the 
two-dimensional  central  limiting  theorem,  since  it  indicates  that  the  components  of 
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the  resultant  are  normally  distributed. 

Let  us  note  that  the  distribution  function  vas  first  obtained  by 

0* 

Rayleigh  precisely  as  a  law  of  the  distribution  of  the  resultant  amplitude  of  the  sum 
of  a  large  number  of  isoperiodic  vibrations  with  random  phases. 

If  the  components  ^  and  of  the  reflected  signals  are  independent  and  are 
normally  distributed  with  zero  means  and  dispersions  of  ^m^,  the  distribution  ^  (r) 
of  the  amplitudes  of  these  signals  will  be  a  Rayleigh  one  (cf.  Section.  5,  Ch,  2) 

»(r)=5-e~"*. 

m2 

Then  the  distribution  of  the  amplitude  of  the  sum  of  n  independent  reflected  signals 
will,  in  accordance  with  (4.40),  be  equal  to 


00  ,00 


'"’4(w)dyj  s/o(r5)d^  = 
=  rje  *  sJff{rs)ds: 


f* 

JL. 


0  «"»» 

i.e.,  precisely  a  Rayleigh  distribution  with  a  dispersion  of  nm.,  equal  to  the  sum  of 


the  dispersions  of  the  amplitudes  of  the  terms.  Since  for  the  Rayleigh  law  of  dis- 
2  _ 

tribution  »  2  ,  the  correction  itself,  depending  on  n  in  formula  (4.45),  turns 

in  this  ease  to  zero. 

The  solution  presented  here  of  the  probleio  of  the  amplitude  distri.bution  of  a 
resultant  may  also  be  obtained  by  the  general  method,  which  is  used  for  proving  the 
tw>-dimensional  central  limiting  theorem,  i.e.,  by  the  resolution  into  a  series  of 
the  characteristic  function  (4,35) 

6.  Law  of  Large  Numbers 

Let  there  be  measured  a  certain  luiknown  physical  quantity  a.  For  increased 
precision  and  reliability  several  measurements  are  usually  made,  and  then  the  a- 
rithmetic  mean  of  the  measurements  is  supposed  to  equal  the  quantity  a.  The  validity 
of  such  a  supposition  is  based  on  the  law  of  large  nxmbers. 

The  results  of  measurements,  which  are  made  independently  of  one  another,  may 
be  regarded  as  the  random  variables  ^n  •  ®®®^ 

F-TS-9811/V  148 


these  random  variables  is  equal  to  the  quantity  a  being  measured,  i,e.. 


(4.46) 


This  signifies  that  the  measurements  are  free  of  ^stematic  errors.  If  the  measure¬ 
ments  are  made  with  a  guarantee  of  any  (albeit  small)  precision,  the  dispersions  of 
the  random  variables^  ^  are  limited 


Afa  {«*}<»*. 


(4.47) 


I«t  us  examine  the  arithmetic  mean  of  the  measurements,  i.e.,  the  arithmetic 
mean  of  the  mutually  independent  random  variables  ^  ^  ^ 

6i  +  ?!  +  •  •  •  +  5ii 


£  = 


(4.48) 


The  mean  value  of  the  arithmetic  mean  of  ^  is  equal  to 


and  the  dispersion  is 


H 


(“t.lt?) 


(4.50) 


It  follows  from  (4.50),  that  the  dispersion  of  the  arithmetic  mean  of  a  sum  is 
not  less  than  n  times  smaller  than  the  greatest  of  the  dispersions  of  the  terms, 
which  indicates  a  reduction  in  the  scattering  of  the  values  of  the  arithmetic  mean 
with  an  increase  of  tw 

The  law  of  large  numbers  asserts  that  with  a  sufficiently  large  number  of  meas- 
J^*^*?*?*  possible,  with  a  probability  close  to  unity,  to  consider  that  the 

arithmetic  mean  of  the  measurement  results  will  be  as  close  as  is  desired  to  the 
true  value  of  the  quantity,  a,  being  measured. 

The  validity  of  the  law  of  large  numbers  nay  be  expected  from  the  inequality 
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(^•50)t  its  rigorous  proof  being  based  on  one  inequality,  credited  to  Chebyshev, 

Let  a  be  the  mean  value  of  the  random  variable  ^  ,  the  distribution  function  of 
iMch  is  W  (x).  Let  us  evaluate  the  probability  of  the  fact  that  the  random  variable 
^  deviates  from  its  mean  value,  j,  at  least  by  tho  arbitrary  positive  number  €  (Fig. 
31,  shaded  region) 


P{\i-^a\>t)=:  J  w{x)dx. 


and  since 


(X  -  a) 


^  1,  then  . 


j  {x  —  cYw{x)dx. 


U— flO 

If  the  limits  of  integration  are  expanded  to  extend  from  -  00  to  -f  oo,  then,  bearing 
In  mind  that  w  (x)  0,  the  size  of  the  integral  cannot  diminish;  then 


P{15-al5s.j<±  = 


Fig.  31*  Probability  of  the  deviation  of  a  random  variable  from  its  mean 
value  by  not  less  than  ^  V4n  bcpial'^  the  diaded  area. 

From  (4,51)  it  follows  that  |  j^»  i»e»»  deviations  of  $ 

from  its  mean,  considerably  exceeding  the  mean-square,  are  improbable. 

Bnplogrlng  the  Chebyshev  Inequality  (4.51),  it  is  possible  to  evaluate  the 
probability  that  the  arithmetic  mean  t  of  the  siua  of  n  mutually  independent  random 
variables  deviates  from  the  mean  value  of  each  of  the  terms  by  more  than  the  arbi¬ 
trary  amount  6  >  0.  Since  in  accordance  with  (4.50)  the  dispersion  of  the  arith- 
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(^.52) 


aetic  mean  does  not  exceed  «  then 

If  the  quantities  and  £  are  fixed,  then  from  (^.52)  there  follows  the  limit¬ 
ing  relationship 

|taP{|!-a)>.}=0, 


The  probability  limit  of  the  converse  inequality  is  equal  to  unity 

lira  P{|5— al<.)==i. 


Formulas  (4,53)  ^d  (4.54)  are  an  analytic  statement  of  the  law  of  large  num¬ 
bers,  liiich  may  be  formulated  in  the  following  manner:  if  the  random  variables  , 

$  A,» « « »  C  n  ftp*  mutually  independent,  have  the  same  mean  values  and  terminal  dis¬ 
persions,  then  the  probability  of  the  declination  of  the  arithmetic  mean  of  these 
random  variables  from  their  mean  value  by  an  amount  greater  than  f >0 ,  tends  toward 
tero  when  n — »  oo  ,  no  matter  how  small  ^  be. 

While  an  individual  random  variable  may  take  values  very  far  from  its  mean  value, 
it  is  practically  certain  that  the  arithmetic  mean  of  a  large  number  of  independent 
random  variables  takes  values  dose  to  the  mean  value  of  the  random  variable.  This 
is  explained  by  the  fact  that  in  the  formation  of  an  arithmetic  mean,  the  random 
deviation  of  the  individual  terms  of  a  sum  from  their  mean  value  cancel  each  other 
out  to  a  considerable  extent. 


7.  Simplest  Problem  of  the  Theory  of  Errors 

The  law  of  large  numbers,  like  the  Lyapunov  theorem,  is  one  to  those  relation¬ 
ships  which  are  valid  for  a  . ivy  large  number  of  summed-up  independent  random  vari¬ 
ables.  For  the  practical  application  of  these  limiting  relationships  it  is  necessary 
to  have  an  estimate  of  the  e*.Tor  in  relation  to  the  number  of  n  items.  A  similar 
estimate  was  made  for  the  central  limiting  theorem  in  Section  2* 

According  to  the  law  of  large  numbers  with  a  very  large  n,  it  is  practically 
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certain  that 


,  {i+€i  +  -£, 

1  = - ;; - ?=feO. 


(4.55) 


where  ^  (k  =  1 ,  2,  . .  .n)  are  mutually  independent  random  variables  which  have  the 
same  mean  values  equal  to  a. 

Let  us  make  it  our  purpose,  with  a  given  small  number  of  n  measurements  to  find 
the  probabilliy  of  the  fact  that  the  deviation  of  the  true  value  of  the  quantity  a, 
undergoing  measurement,  from  the  ai*ltfametic  mean  ^  of  the  esqperimental  data  will  not 
exceed  a  certain  fixed  £  >0  ,  i.e., 

‘  P{~€<£-a<.}  =  P{£_t<a<t+t}=«. 


(4.56) 


Ihe  probability  is  sailed  the  reliability,  and  the  quantity  S  ,  the  precision 
of  the  approximate  equality  (4.55)  with  a  given  number  of  n  measurements.  The  Inter¬ 
val  (^-£  ,^+£)  idiich  contains  the  true  value  of  the  quantity  a^  being  measured, 
is  called  the  confidence  Interval. 

The  simplest  problem  of  the  theory  of  errors  consists  in  estimating  the  pre¬ 
cision  (the  length  of  the  confidence  interval)  and  the  reliability  of  the  approximate 
equality  (4.55)  for  a  small  number  of  n  measurements. 

if  the  precision  £  is  given  (i.e.,  the  length  of  the  confidence  interval), 
then  according  to  the  law  of  large  numbers  the  reliability  of  the  measurements  will 
grow  as  the  number  of  measurements  increases,  approaching  unity.  With  a  given  num¬ 
ber  of  measurements  the  reliability  will  be  the  greater,  the  wider  is  the  confidence 
Interval,  i.e.,  the  greater  is  the  permissible  error. 

Li  other  words,  with  a  given  number  of  measurements  it  is  impossible  to  Increase 
reliability  without  reducing  precision  or,  conversely,  to  increase  precision  without 
reducing  reliability. 

Ihe  objective  of  the  error  theory  is  to  find  a  link  between  the  quantities  of 
reliability  oc  ,  precision  £  ,  and  the  number  of  measurements  n,  whldi  would  make 
it  possible  on  the  basis  of  a  given  two  of  these  quantities  to  find  the  third. 
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T^e  overall  error  of  physical  weasurenent  naj'  be  regarded  as  the  sun  of  a  large 
number  of  mutually  independent  elementary  errors.  Therefore  the  overall  error  must 
according  to  the  lyapunov  theorem  be  normally  distributed.  The  assumption  that  the 
measurement  results,^  of  the  quantity,  jl,  are  subject  to  the  normal  law  of  distri¬ 
bution 


is  the  basis  of  error  theory. 

2 


If  the  dispersion  cr  of  the  measurement  results  were  known  a  priori  ,  the 
question  of  estimating  the  reliability  and  precision  would  be  solved  very  simply. 

For  instance,  it  could  be  asserted  with  practical  certainty,  i.e.,  with  a  probability 
of  Oi  s  0.997,  that  the  true  valut  of  the  quantity  being  measured  differs  from  the 
arithmetic  mean  by  rio  more  than  3  o'  . 

Let  us  take,  as  the  approximate  value  of  the  dispersion  of  any  of  the  random 
variables,  ^  j,,  the  quantity 


('*.58) 


Then  the  approximate  value  of  the  dispersion  of  the  arithmetic  mean  of  the  independ¬ 
ent  random  variables  ^ .  will  be 


(4.59) 


*-i 


The  random  variable  7)  will,  in  accordance  with  the  results  of  Section 9  of  Ch.3..  ^ 

2  ' 
an  X  type  distribution. 


It  is  natural  to  suppose  that  the  deviation  ^  -a  is  proportional  to  the  root 
of  the  dispersion'll^,  i.e.,^-Q.  =Xt. 

The  random  variable 

’I  (4.60) 
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is  the  normalised  error  of  measurement.  This  random  variable  is  equal  to  the  ratio 


of  two  random  variables,  one  of  which  is  distributed  normally  (with  a  zero  mean), 

. .  vy2 . 


while  the  distribution  of  the  other  is  linked  with  the  ^  distribution. 


Dnploying  the  formulas  of  Section  3  .Ch,3#lt,is  possible  to  shoir  that  when  5^2 


the  distribution  function  ^(x)  of  the  random  variable  is  equal  to 


(^.61) 


Bie  distribution  given  by  function  (4.61)  is  known  by  the  name  of  the  Student  dis¬ 


tribution  or  the  t-distribution. 


For  large  values  of  n,  the  distribution  of  normalized  error,  in  full  accordance 


with  the  lyapunov  theorem,  approaches  the  normal 


UmS„(x)  =  -^e 


(4.62) 


However,  for  small  values  of  n,  the  Student  distribution  differs  markedly  from  the 
liiniting  normal  one.  Fig.  32  shows  the  curve  of  distribution  (4.61)  for  n  =  4  and 


the  curve  of  the  limiting  normal  distribution  (4.62).  It  follows  from  the  equations 


of  these  curves  that  the  probabilities  of  a  large  deviation  are  greater  for  the  ran¬ 


dom  variable  than  for  a  normally  distributed  random  variable. 


'jJormal  laif- 


//  \\ 

r  \\  Student 

/  distribution 


t  t  S 


Pig.  32.  Probability  density  of  normal  distribution  and  of  student 


disttibution  wiien  n  =  4 
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The  integral  distribution  function,  i.e,,  the  probability  that  t  will  not  ex¬ 
ceed  the  quantity  x,  is  equal  to 


M 

P{t<x)=  J  S,(x)</x. 


(^.63) 


When  n — >oo,  as  follows  from  (4.62),  the  integral  j  ^  J(?)*  Therefore 

—oa  ~ 

the  Laplace  function  F(x)  is  sometimes  called  the  Integral  of  error  probability. 

Returning  to  the  relationship  (4.56),  we  introduce  the  quantity  t^t,  proportional 
to  an  error  of  S  with  a  given  reliability  of  of 

•=V..  (4.64) 


Then  the  inequality 


—#<1— 0<t 


(4.65) 


corresponds  uniquely  to  the  inequality 


(4.66) 


Therefore  the  probability,  that  the  quantity  a  being  measured  lies  within  the  con¬ 
fidence  interval  (4.65),  is  equal  in  accordance  with  (4.56)  to 

=  j  S„{x)dx  =  a, 


and,  since  the  Student  function  is  even. 


PU-«<c<£+t}=2|s,(jc)rfx  =  a. 


(4.67) 


Equality  (4.6?)  establishes  the  desired  dependence  between  the  quantities  oC  ,  £  ,and 
n,  i.e.,  between  reliability,  precision,  and  the  number  of  measurements. 

'Appendix  III  contains  a  table  of  the  values  of  toe.  which  satisfy  (4.6?)  for 
given  instances  of  oc  and  -n. .  The  last  line  of  the  table  (n  =  oo )  corresponds  to  a 
normal  distribution  of  error. 
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The  magnitude,  tg^,  is  the  proportionality  factor  betvfeen  the  length  of  the 
confidence  interval  (precision),  and  the  mean-square  deviation  of  the  results  of 
measurement. 

With  a  given  reliability  oc  the  smallest  t  .  (i.e.,  the  smallest  length  of  the 
confidence  interval)  corresponds  to  a  normal  distribution  of  error.  Thus, for  instance 
in  Section  3  of  Ch.  2  it  was  pointed  out  that  for  a  normally  distributed  random 
variable  it  is  practically  certain  (with  a  probability  of  0.997)  that  none  of  its 
values  will  deviate  by  more  tha  3  f rom  the  mean,  i.e.,  that  the  length  of  the  con¬ 

fidence  interval  is  equal  to  6  "7^.  Actually,  particularly  with  a  small  number  of 
measurements,  the  length  of  this  interval,  corresponding  to  a  practical  certainty, 
will  always  be  greater. 

8.  Determination  of  Distance  by  Independent  Obseirvers 

As  an  illustration  of  the  statistical  method,  presented  above,  of  processing 
measurement  results  let  us  examine  an  example  connected  with  determining  the  distance 
of  a  certain  object  by  means  of  radar.  To  increase  reliability,  the  indicator  is 
observed  by  several  independent  obsei-vers,  and  then  the  arithmetic  mean  of  the  ob¬ 
served  distances  is  taken  as  the  true  value  of  the  distance. 

Let  there  be  the  following  five  distance  measurements,  made  by  five  independent 
observers ; 

^  ^  =  39.1km;  ^2  “  ^3  ~  ~  *^5  ~ 

The  arithmetic  mean  of  these  observations  yields  the  distance  of 
^  _  39.1  +  39,7  +  38,3  -f  39,6  +  38,8  _  jg  j  ^ 

The  approximate  dispersion  of  the  arithmetic  mean  of  the  observations  is,  in 
accordance  with  (4.59),  equal  to 

^’ = 4?!  (0*  +  0-6*  +  0.8*  +  0.5*  +  0,3*)  =  =0.067, 


^  =  0.26  km. 
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Let  us  assign  a  reliability  of  oC  =  0,99  and  estimate  the  precision  of  the  dis¬ 
tance  determination.  In  the  table  in  Appendix  III,  for  oC  =:  0.99  and  n  =  5  we  find 
that  toe  =  i«e.,  the  confidence  interval  extends  in  each  direction  from  the 

arithmetic  mean  for  an  extent  equal  to  almost  five  mean-square  errors.  Consequently, 
it  is  possible  to  assert  with  a  probability  of  0,99  that  the  true  distance  from  the 
object  lies  within  the  limits  of  from  ^  -  tgp^  -  39*1  +  4,604  •  0.26  =  37.91  Im  to^-l- 


•f  toiyi  »39*1  li»60U«0*26  »  Uo«29  Ion,  i*e*,  the  distance  is  determined  with  an  error 

not  greater  than  t  =  1,19  km,  which  constitutes  35^.  In  order  to  have  a  narrovrer  con¬ 
fidence  intearval  with  the  sane  reliability,  five  observers  are  insufficient. 

If  normal  distribution  had  been  used  for  measuring  the  error,  instead  of  the 
Student  distribution,  it  would  have  been  practically  certain  that  the  true  distance 
lies  within  the  limits  of  -  37^  -  39.1  -  3*0.26  =  38,32  km  to  4  +  39*1  + 


3*0,26  =  39«88  km,  which  corresponds  to  an  error  of  only  780  m  or  2>b, 

It  is  possible  to  solve  a  problem  of  another  type;  with  an  assigned  permissible 
error  of,  say  0,5^  to  find  the  probability  that  the  distance,  found  as  the  arithme¬ 


tic  mean  of  5  independent  observations,  will  differ  from  the  true  distance  by  no  more 


than  by  195  m  (idiich  constitutes  about  0.555  of  the  distance  measured.)  Then  = 

4  0.005  *  39.1  •[ 

=  0.26  =  0.75,  and  by  the  table  in  Appendix  III,  after  linear  extrap¬ 

olation,  we  find  for  n  =  5  the  desired  probability  of  =  O.504. 


Thus,  from  the  results  of  five  independent  measurements  it  is  impossible,  with 
an  error. not  exceeding  195  n  (0.5^).  to  form  a  sufficiently  reliable  judgment  as  to 
the  distance  measured,  while  for  precision  on  the  order  of  1  km  (35o),  this  number  of 
observations  was  entirely  sufficient. 
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Chapter  V 
RANDOM  PROCESSES 


1«  Definitions  and  Terminology 

The  preceding  chapters  ,  ‘-isonted  the  basic  rules  of  classical  probability  theory, 
as  well  as  examples  illustrating  the  applications  of  this  theoiy  to  the  solution  of 
certain  technical  problems.  A  large  part  of  this  theory  is  devoted  to  the  study  of 
the  laws  of  distribution  of  one  random  variable  or  of  a  system  of  a  finite  number  of 
random  variables.  Ihe  following  chapters  will  be  devoted  to  the  exposition  of  certain 
results,  and  applications  to  radio  engineering,  of  the  modern  theory  of  random  proces¬ 
ses  —  of  a  theory  •kdilch  deals  with  random  variables  which  depend  on  a  continuously 
changing  parameter,  time. 

The  best-known  example  of  a  random  process,  to  which  in  the  course  of  our  pres¬ 
entation  ve  shall  be  constantly  returning,  are  fluctuation  (shot  and  thermal)  noises 
in  radio  devices  (both  of  the  transmitting  and  the  receiving  kinds).  Thus,  in  the 
observation  of  noise  voltage  at  the  output  of  identical  instruments,  it  appears  that 
the  forms  of  this  voltage,  or  the  time  functions  describing  this  random  process,  dif¬ 
fer,  The  task  of  the  theory  of  random  processes  lies  in  finding  the  statistical  laws 
governing  the  various  functions  which  describe  the  same  plysical  phenomenon  (fluctu¬ 
ation  noise),  analogously  to  the  manner  in  which  the  classical  probability  theory 
establishes  statistical  laws  by  the  observation  of  random  variables. 

Quantitatively,  a  random  process  is  described  as  a  random  function  of  time (t), 
the  values  of  which  are  random  variables  at  any  moment  in  time* 

Just  as  the  random  variable^  is  characterised  by  an  aggregate  of  possible  values 
and  by  a  distribution  of  the  probabilities  that  these  values  will  occur  so  the  random 
process  $  (t)  (a  random  function  of  time)  is  defined  as  an  aggregate  of  time  functions, 
together  with  the  laws  characterizing  the  statistical  properties  of  this  aggregate. 

Each  of  the  functions  of  this  aggregate  is  called  the  realization  of  a  random  func¬ 
tion. 


F-TS-98il/V 


time  instants*  t^  and  i2*  wo  again  examine  N  realizations  of  a  random 

function  and  isolate  from  this  number  those  22  realizations,  the  values  of  which  at 
the  time  instant  t  =  t^  are  smaller  than  jcy  and  at  the  time  instant  t  =  tg  are 
smaller  than  X2« 

It  is  possible  to  assume,  analogous  to  the  preceding  case,  that  with  a  suf¬ 
ficiently  large  N,  the  relative  portion  of  the  functions  at  t  *  t*  lying  below 
the  level  of  x^,  and  at  1  «  t2  below  the  level  of  x,,  will  possess  statistical 
stability,  i.e.,  will  remain  an  approximately  constant  number.  This  number  is  called 
the  probability  that  at  t  =  t ^  the  random  function  ^  (t)  lies  below  the  level  of  x^  • 
and  at  t.  =  t2  below  the  level  of  X2, 

The  Indicated  probability  P {  ^  (t^)  <  x^,  ^  (tg)  <  Xg}  is  a  function  of  the 
four  variables  x^ ,  X2,  t^ ,  t2 

Xj, ^{(2)^X2}  /- 


and  is  called  the  second  Integral  probabilit'/^  distribution  function  of  a  random 


process. 


If  the  function  FgCj^.  22*  tg)  has  a  derivative 

I  (*|.  Xj-  ^1*  ^1) _ V  t  t\ 

- - X2,  rj. 


(5.45 


thio  derivative  is  called  the  two-dimensional  probability  density  or  the  second  dis¬ 
tribution  function.  Two-dimensional  distribution  functions  are  statistical  charac- 
texd.sties  of  the  random  process,  on  the  basis  of  which  it  is  possible  to  evaluate  the 
relationship  between  the  probable  values  of  a  random  function  at  two  arbitrary  tine 
instants. 


In  a  similar  manner  it  is  also  possible  to  determine  the  probability  that  the 

random  fUncUon  ^  (t)  will,  at  n  instants  of  1^,  lie  below  the  levels  of 

il»  J?2****2n  *‘‘’SP«ctively  ^  ■ 

-  '  ... 

Xjt ...  x^,  (2,  (5.5) 
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The  indicated  probability  is  a  function  of  2  n  variables  and  is  called  the  n-th 
integral  probability  distribution  function  of  a  random  process.  If  Xjj  22****-n» 
lit  t2*...tj^)  has  a  derivative 

d^Fff  (j(j>  Xj, ...  x^,  ti,  /j, ...  /^) 


<txi<ixj.^dx„ 


‘  ^3'  •••  ^n’  ^2>  •••  ^b)> 


(5.6) 


this  derivative  is  called  the  n«dlnensional  probability  density  or  the  .n~th  dis¬ 
tribution  function. 

The  sequence  cf  distribution  functions  w^(x^.  t^).  ¥2^2?^*  ^2*  -1  ^2^’*** 

Wq(x^,  ?2****-n*  -1*  ^2****-n^  constitute  a  unique  ladder,  in  the  ascent  of  vhich 
more  detailed  fainiliarity  with  the  random  process  is  acquired*.  Knowing  the  first  n 
distribution  functions,  it  is  possible  to  evaluate  the  relationship  between  the 
probable  values  of  a  random  function  at  n  arbitrary  time  instants.  The  distribution- 
function  sequence  in  question  must  possess  all  the  properties  of  probability  dis¬ 
tribution  ftinctlons. 

Specifically,  from  a  distribution  function  cf  the  n-th  order  it  is  possible  to 
obtain  all  the  distribution  functions  of  the  lower  order,  through  to  the  first  [cf, 

(2.47)]  . 

Thus,  the  statistical  propertie.s  of  a  random  process  (of  a  random  function)  may 
be  characterized  by  means  of  an  n-dinensional  distribution  function,  and  the  more  ac¬ 
curately  so  the  greater  is  the  number  of  n. 

limiting  ourselves  to  the  n-dimensional  distribution  function  of  a  random  process, 
w*  essentially  identify  the  random  function  with  the  aggregate  of  n  random  variables 
^  (ll)»  ^  (tj),...  §  (tp)  —  values  of  the  random  function ^(t)  at  the  fixed  time 
instants  t^,  t^. . » , t^. 


♦  Of  course,  the  statistical  properties  of  a  random  process  nay  be  described  by  the 

linked 


sequence  of  characteristic  functions  G,  (v-,  ,t  ),  0 T  ,  t^) . linl 

by  Fourier  transformations  to  the  distribution  functions  )*  ,  X 

(cf.  Section  7,  Ch.m),  ‘  ^  ^ 
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For  a  graphic  illustration  of  the  concepts  introduced  [here]  ,  we  turn  to  a 


geometric  representation  of  an  aggregate  of  realizations  of  the  random  function  ^  (t) 


in  the  form  of  the  set  of  plane  curves  ^  ^^^(t),  (t),,,,  <5  ••♦  (^^2*  33X 


(2)< 


M, 


wpy. 


Fig.  33.  Realization  aggregate  of  a  random  function. 

/v) 

Let  us  now  replace  each  of  the  $  '  '(t)  curves  by  a  sequence  of  rectangular 
pulses  of  the  same  width  T,  equal  to  the  period  of  their  repetition,  The  amplitude 
of  any  m-th  pulse  of  this  sequence  is  selected  as  being  equal  (Fig.  3^)  ^ 


Let  us  call  the  sequence  fixed  in  this  manner,  a  sequence  of  br.se  pulses*. 


Fig.  3^.  Sequence  of  base  pulses. 


*  The  replacement  of  a  process  realization  by  a  base  pulse  sequence  is  known  in 
communications  practice  as  the  time  quantization  (breaking-up  into  parts)  of  a 

signal. 
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If  the  curve  ^  is  reproduced  ty  a  radio  system  with  a  pass  band  A  , 

then,  if  the  width  of  the  base  pulse  satisfies  the  inequality 


T  «  T  . 

the  reproduction  of  ^  by  the  system  is  practically  equivalent  to  the  repro¬ 

duction  of  the  base  pulse  sequence,  since  in  a  system  with  the  indicatea  pass  band 
it  is  impossible  to  distinguish  a  charige  in  the  '{t)  ciirve  within  an  interval  of 
tine  shorter  than  T.  Replacing  each  of  the  realizations  ^  '(t)  iy  a  sequence  of 

base  pulses,  we  obtain  an  aggregate  of  sequences  of  rectangular  pulses  occurring  in 
instants  of  tine,  representing  multiples  of  their  duration,  the  amplitude  of  an  im¬ 
pulse  occurring  at  the  instant  -x.  T ,  being  capable  of  taking  one  of  the  values  of 

^2*  *  *  *  Abji.»  •  •  •  • 

An  aggregate  of  these  amplitude  values  may  be  treated  as  the  random  variable  ^  (t^), 
which  characterizes  the  random  amplitude  of  a  pulse  occurring  at  the  instant  tjj  =  mT, 

Thus  the  random  process  $  (t)  Is  approximately  represented  in  the  form  of  a 
sequence  of  base  pulses  with  a  random  amplitude  of  ^  this  approximation  being  the 
more  precise,  the  shorter  the  duration  of  the  base  pulse. 

It  follows  from  iriiat  has  been  said,  that  a  one-dimensional  distribution  function 
of  a  ra.idoEt  process  represents  the  probability  density  of  the  random  amplitudes  of 
base  pulses,  which  may  depend  on  the  number  of  the  pulse  (i.e.,  on  time).  A  two- 
dimensional  distribution  function  of  a  random  process  yields  the  joint  distribution 
of  the  random  amplitudes  of  any  pair  of  base  pulses.  Analogously,  an  n-dimensional 
distribution  function  yields  the  joint  distribution  of  ary  group  consisting  of  n 
base  pulses. 

The  simplest  type  of  a  random  process  will  be  one  in  which  the  randcn  amplitudes 
of  the  base  pulses  are  mutually  independent.  Such  a  random  process  is  fully  charac¬ 
terized  by  a  one-dimensional  distribution  function.  In  the  general  case,  the  ampli¬ 
tudes  of  the  base  pulses  occurring  at  different  points  of  time  will  be  dependent,  and 
for  a  complete  characterization  of  a  group  of  n  base  pulses  it  will  be  necessary  to 
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assign  an  n-dlnensional  distribution  function  to  their  random  amplitudes. 

Very  frequently,  the  random  variable  ^  (t)  nay  contain,  as  an  item,  a  given 
function  of  tine  S(t) 

w)=m-^s{t). 

(5.7) 

Let  us  agree  to  say  that  (t)  and  S(t)  represent  respectively,  the  purely 
random  and  the  determined  parts  of  process  $  (t).  It  is  clear  that  the  indicated 
parts  of  a  process  are  always  independent. 

Since  the  probability  density  for  S(t)  is  represented  by  the  delta«function 
5^[  X  -  S(t^ (cf. Section 3,  Ch«II),the  distribution  function  of  sum  (5.7)  is,  in  ac¬ 
cordance  with  (3*26)  (cf.  also  Appendix  IV) 


{x.  t)  =  J*  tt'i  {«,  /)  0  [x  —  S  (0  —  «1  du  ~ 

~”=Wi[x-S{t),  t]. 


(5.8) 


vhere  w^(x,t)  is  the  distribution  function  of  the  purely  random  part  of  the  process. 
Analogously,  the  two-dimensional  distritwtion  function  of  sum  (5.7)  i-** 


Ur2(X|,  X.j,  ..  t.,)  —  —  ■5(^2),  ^2l» 


(5.8») 


where  V2(x^,  X2,  t^,  tg)  is  the  two-dimensional  distribution  function  of  ^^(t). 

In  many  radio-engineering  problems,  the  subjects  of  investigation  are  the  useful 
signals  and  the  accompanying  interference,  the  aggregates  of  >hich  constitute  a 
random  process.  The  signal  is  often  regarded  as  the  determined  part  of  this  process, 
and  the  interference  as  the  purely  random  part.  However,  in  the  modem  general  theory 
of  communications  (information  theory),  the  signals  are  regarded  not  as  given 
functions  of  time,  but  as  an  aggregate  of  possible  functions  of  time,  possessing 
definite  statistical  characteristics,  i.e.,  as  a  purely  random  process.  On  the  other 
hand,  problems  are  possible  where  interference  (for  instance,  artificially  created) 
becomes  the  determined  part  of  the  process.  The  expressions  for  the  distribution 
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functions  of  signals  Jointly  with  noise  will  depend  on  what  assunptions  are  nade 
concerning  the  nature  of  each  of  the  indicated  processes. 

2*  Stationarity 


Me  now  pass  to  a  presentatloQ  of  the  properties  of  the  most  Liqx>rtant  class  of 
random  processes  —  the  so-called  stationary  random  processes. 

The  random  process  $  (t)  is  called  stationary  if  its  distribution  function 
Wn(x^»  *2,  ^»»».t^).  of  an  arbitrary  n  order,  does  not  diange  with  aiy 

shift  of  the  entire  group  of  points  t^,  t2,...tj^  along  the  tine  axis,  i.e,,  if  for 


ary  n  and  V 


>^2.- -  -V  = 


(5.9) 


In  other  words,  a  random  process  will  be  stationary  when  the  expressions  for  a 
distribution  function  of  any  order  do  not  depend  on  the  position  of  the  starting 
point  of  the  time  reckoning. 

It  follows  from  this  definition,  that  for  a  stationary  random  process: 

a)  the  first  distribution  function  has  the  same  form  at  any  instant  in  time 
(I.e.,  does  not  depend  on  time) 


(5.10) 


b)  the  second  distribution  function  may  depend  only  on  the  difference  t^  • 


Wa(X,,  Xj,  /2)  =  »2(*1.  ^2.  t2~fi). 


(5.11) 


c)  the  third  distribution  function  may  depend  only  on  the  two  differences 
t^  •  tf  and  t^  > 


Xj,  /j,  /j)  —  Vj(X|,  Xj,  X,,  Ij—ti,  — '/|) 


(5.12) 


In  the  representation  of  a  random  process  ty  a  sequence  of  base  pulses  the  above 
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properties  of  a  stationary  process  will  signify  thats  l)  the  amplitude  distribution 
functions  of  all  base  pulses  are  the  same;  2)  the  joint  distribution  't  the  random 
amplitudes  of  any  pair  of  base  pulses  depends  only  on  the  distance  between  the  pulses, 
and  does  not  depend  on  the  location  of  this  pair  on  the  time  axis;  3)  the  joint  dis¬ 
tribution  of  any  triplet  of  base  pulses  depends  only  on  the  distances  between  one 
pulse  and  the  two  others,  and  does  not  depend  on  the  loca+ion  of  these  pulses  on  the 
time  axis,  etc. 

In  a  number  of  problems,  it  suffices  to  know  less  about  a  random  process  than  is 
disclosed  by  the  distribution  functions;  it  Is,  for  instance,  possible  to  limit  one’s 
self  to  such  summary  rumerlcal  characteristics  of  a  process  as  the  initial  moments 
of  distribution,  which  in  the  genera^,  case,  are  determined  by  means  of  (2.96)  in  the 
following  manner; 


:J  I  ...  J 


» times 

X  (^1.  -^2. . . .  /„  /j, . . .  l„)dxidxj . . .  dx^. 


(5.13) 


These  distilbution  moments  may  also  be  determined  with  the  aid  of  character¬ 
istic  functions  by  means  of  formula  (3.73). 

The  simplest  numerical  characteristics  of  a  random  process  are  the  distribution 
moments  of  the  first  two  orders*. 

The  mean  value  of  the  ranoyo  process  (or  the  first  moment  of  the  one-dimensional 
law  of  distribution) 


"*1  {?  (0}  =  J  {x,t)dx=a  [t)\ 


(5.14) 


the  dlspersi.on  of  the  random  process  (or  the  second  central  moment  of  the  one- 
di.T'.i.slonal  law  of  distribution) 


m,  {(5  (/)  -  aj2}  =  J  (x  -  ay  w,  (x.  t)  dx  =  (/); 


(5.15) 
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the  correlation  function  of  the  random  process  (or  the  nixed,  second  initial 
moment  of  the  two-dimension  law  of  distribution) 


0»  00 


"»l  (U^)«(/i))  =  jj  ■^l^2«'2(*l.  X2.  /,.  /j)  dX,f/X2,: 


(5.16) 


=  B(/,./2). 

If  the  two-dimensional  characteristic  function  ®  ^^  )  of  a  random 

2  1  2  1  2 

process  is  known,  the  correlation  function  can  be  found,  in  a  jordance  with  (3.78), 
by  moans  of  the  formula 


(5.16*) 


It  follows  from  (5*10)  that,  for  a  stationary  random  process,  its  mean  value  a 


and  dispersion  are  constants,  not  depending  on  time. 


Since  the  second  distribution  function  W2(x^,  x^,  t^  -  of  a  random  station¬ 


ary  process  depends  only  on  the  difference “  ^2  *  ^1*  correlation 

function  of  such  a  process  also  depends  only  on  the  variable's' 


M  00 


— /,)=  J  Xj,  f3  —  f,)dx,dx2. 


(5.17) 


Vftiile  the  experimental  determination  of  the  two-dimensional  distribution  function 
of  a  random  process  constitutes  a  cum' ersorae  and  in  practice  rarely  achievable  task, 
the  experimental  determination  of  the  correlation  function  of  a  stationary  random 
process  can,  as  will  be  shown  below,  be  accomplished  with  comparative  ease.  For  a 
very  large  number  of  problems  the  correlation  function  is  a  sufficiently  complete 
statistical  characteristic  of  a  stationary  random  process. 

The  part  of  theory  devoted  to  the  study  of  only  those  properties  of  random 
processes,  which  are  determined  by  their  moments  of  the  first  two  orders,  is  called 
the  corirelatlonal  theory  of  random  processes.  Since  multidimensional  distributions 
have  no  place  in  correlational  theory,  it  is  natural  within  the  framework  of  this 
theory  to  consider  as  stationary  all  random  processes  whose  mean  value  and  dispersion 
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do  i>3t  depend  on  time,  and  tdiose  correlation  function  depends  only  on  the  time 
difference  of  T  =  ^2  "  ^1*  processes  satisfying  these  conditions  arc  called 

stationary  in  the  broad  sense  (or  stationaiy  in  the  sense  of  A.  Ya.  Khinshin). 

Stationarity  in  the  broad  sense  is  not  identical  with  the  precise  definition  of 
stationarlty  in  accordance  with  (5»9).  Random  processes  which  are  stationary  ac¬ 
cording  to  (5.9)  will,  of  course,  always  be  stationary  in  the  broad  sense,  but  not 
▼ice  versa. 

Within  the  framework  of  the  correlational  theory,  it  is  clearly  sufficient  to 
know  the  random-process  distribution  function  of  an  order  no  higher  than  the  second. 

It  should  be  noted,  that  the  two-dimensional  distribution  function,  and  even 
more  so,  the  correlation  function,  cannot  describe  a  random  process  in  as  much  detail 
as  can  multi-dimensional  distribution  functions.  Moreover,  the  same  correlation 
function  can  correspond  to  different  processes,  in  other  words,  an  equality  of  cor¬ 
relation  functions  tioes  not  signify  an  Identity  of  processes.  However,  at  present 
only  the  correlational  theory  of  random  processes  has  been  developed  to  the  extent 
that  a  widespread  application  of  it  has  become  possible  in  various  engineering 
problems.  The  practical  value  of  the  correlational  theory  is  increased  ty  virtue  of 
the  fact  that  a  large  class  of  stationary  ranioc.  processes  —  that  of  the  so-called 
normal  random  processes  —  is  fully  uwfined  by  the  correlation  function  alone.  For 
these  processes  the  concepts  of  stationarity  in  the  pz^cise  and  in  the  broad  senses 
coincide. 

The  random  processes  represented  by  random  functions  of  a  continuously  changing 
argument  t,  which  are  frequently  encountered  in  radio-engineering  problems,  do  not, 
however,  completely  exhaust  the  degiand?  of  practical  application. 

.Sometimes  a  random  process  is  thought  of  as  broken  down  into  consecutive  steps, 
i.e.,  is  represented  by  random  functions  of  a  discrete  time  t.  Such  a  process  was 
the  sequence  of  base  pulses,  whose  amplitudes  represented  the  aggregate  of  the  random 
variables  which  depend  on  the  tine  instant  t^  (n  *  -2,  -1,  0,  1,  2,...).  Another 
example  of  a  random  process  with  discrete  time  is  the  simple  Markov  chain  (cf. Section  10, 
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Ch,  1),  This  chain  possesses  the  special  property,  that  the  probability  of  a  tran¬ 
sition  from  the  state  of  x(s)  to  one  of  the  possible  states  of  ^  when  t  =  s  +  1  does 
not  depend  on  what  states  occurred  at  the  instants,  t  ^  s.  Random  processes  pos¬ 
sessing  the  indicated  property  are  called  processes  without  aftereffect.  The  absence 
of  an  aftereffect  indicates  that  the  statistical  interrelationships  in  a  random 
process  with  discrete  time  do  not  extend  further  than  for  one  step. 

If  in  the  Markov  chains  the  limitation  of  the  values  of  t  to  >diole  numbers  is 
discarded,  we  obtain  the  general  Markov  process,  directed  by  the  probability  of  the 
transition  of  p{ro,x  ,  j  from  the  state  of  x  at  'the  Instants  to  one  of  the 
aggregate  of  states  of  ^  at  the  instant  t.  The  general  Markov  process  is  also  a 
random  process  with  discrete  time  without  aftereffect. 

An  exposition  of  the  theory  of  Markov  processes  goes  beyond  the  bounds  of  the 
present  book.  In  the  future  we  shall  deal  primarily  with  such  random  processes  with 
continuous  time,  for  which  a  statistical  link  between  past  states  and  future  ones  is 
essential. 


J,  Ergodicity  of  Stationary  Processes 


Certain  random  processes  possess  a  property  important  to  practical  applications 
of  the  theory  under  discussion,  namely  ergodicity.  Without  going  into  the  fine  points 
of  strict  definitions,  let  us  note  that  a  random  process  is  crgodic,  if  any  of  its 
realisations  has  the  sane  statistical  properties,  i.e.,  if  with  the  passage  of  time 
a  given  realisation  undergoes  on  the  average  the  sane  changes  that  can  affect  any 
other  realisation  of  the  random  process.  Therefore,  in  the  case  of  ergodic  random 
processes,  any  statistical  characteristic  obtained  by  the  averaging  of  an  aggregate 
of  possible  realisations  can,  with  a  probability  of  as  close  as  desired  to  unity,  be 
obtained  by  the  averaging  a  single  realization  of  a  random  process  over  a  sufficiently 
large  Interval, 

Let  us  observe  over  a  sufficiently  long  period  of  time,  T,  some  realisation 
(k) 

^  (t)  of  a  random  process  (for  instance,  the  noise  voltage  at  the  output  of  some 
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receiving  set).  Let,  for  the  time  T,  the  total  time  that  the  realization  (the  noise 
voltage)  stays  within  the  definite  limits  of  from  u^  to  Ug  (Fig.  35)  be  equal  to 

/<*»(r)  =  A/,  +  A/,  +  ...+A/«. 

The  ratio  y^(T)  =  ■-'  ^^  is  called  the  relative  time  of  dwell  of  realization 

_  Ihl  .  ' 

^  \t)  within  the  limits  of  from  to  u^. 


Fig.  35.  Total  time  of  dwell  of  a  realizati  a 
within  the  limits  of  u^,  u^. 

Let  us,  for  example,  examine  the  position  of  a  material  point,  moving  in  ac¬ 
cordance  with  the  law  oCt^-  a  sin  d)  6  ,  at  a  random  time  instant  t  =  $  .  The  rela¬ 
tive  time  of  dwell  of  this  point  below  the  level  of  x(-a^xCa)  is  equal  (Fig.  36) 
to 


F{x) 


.  .  /'2«  ,  \  •  *  2*  / 
_ sin  J  +  ~~[ 


n 

I*  » 


I  X  ^ 

arc  sin  —J 


M 


2k 


or 


^(*)=4+7arcsin-5 ,  |xl<c. 


Fig,  36.  Position,  at  a  random  time  instant,  of  a  point 
moving  according  to  harmonic  law. 

In  the  example  under  examination  the  relative  time  of  dwell  simply  coincides 
with  the  integral  distribution  function  of  the  random  variable]^  jcompare  (3.17)]  . 
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If  the  random  process,  ^  (t)  is  ergodtc,  given  large  values  of  T  it  is  possible 
to  consider  that  the  relative  time  of  dwell  Vi.(T),  coincides  with  the  probability 

f\ 

p{u^<  5  (t)  v<  Ug}  of  the  fact  that  the  values  of  a  random  function  are  bounded 
by  the  limits  of  <  $  (t)  u^.  This  probability  has  been  defined  above  as  the 
share  of  the  realizations  satisfying  a  given  condition.  Since  »>  (T)  is  a  random 

K 

variable,  its  proxim'  y  to  the  constant  P,  at  large  values  of  t  may  be  asserted  only 
with  some  (sufficiently  great)  probability.  Vfe  have  already  encountered  a  similar 
circumstance  in  Chapter  IV  (the  law  of  large  numbers),  ^ere  it  was  shown  that  the 
artthmatic  mean  of  a  large  number  of  independent  random  measurements  may,  with  a 
probability  of  close  to  unity,  be  considered  as  the  mean  value  a  of  a  random  variable 
(which  is  obtained  ly  averaging  the  aggregate  of  its  possible  values  a  =  I  xw(x)  dx). 

OO 

Therefore,  the  equivalence  of  the  two  methods  of  averaging  ergodic  random  processes  — 
averaging  an  aggregate  of  realizations  and  the  averaging  of  a  single  realization  with 
respect  to  time  —  is  also  sometimes  called  the  law  of  large  numbers. 

The  soviet  mathematician  A,  Ta,  Khinchin  has  proved  an  important  theorem  to  the 
effect  that  stationary  random  processes  (with  sufficiently  general  assumptions  [zl) 
are  ergodic.  From  this  ergodic  theorem  there  directly  follows  the  possibility,  in 
the  computation  of  such  important  characteristics  of  the  stationary  random  process 
as  the  distribution  moments  of  (t)}  and  the  correlation  function  B('^)  =  m^. 

■^(t)^  (t  +  (rrhich  are  averages  for  an  aggregate  of  realizations),  of  restricting 
one’ s  self  to  only  one  single  realisation  of  the  random  process,  naricly 
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dt. 


(5.t8) 


{5(/)l(< 4-*)}  =lim  ^  f  5<'»(0r(/  +  T)d/ 


(5.19) 

independently  of  r,  i.e, ,  of  irtilch  of  the  realizations  is  being  averaged  with  respect 
to  time.  An  averaging  of  the  product  4  (^^  ^(t+v)  with  respect  to  time  is  called 
the  autocorrelation  function  of  process  5 (t), 
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From  (5.18),  when  k  «  1,  we  find  the  mean  value  of  a  stationary  random  process 


# 

T-*v>  •  J 


(5.20) 


idiich  nay  thus  be  treated  as  a  constant  component  of  this  process* 

The  second-order  initial  moment  equal  to  the  value  of  the  correlation  function, 
when  Y  =  0,  is  obtained  from  (5*18),  when  k  =  2 


/Mj  =  B  (0)  =jhn  Y  j  if » (Ol^dL 


(5.21) 


If  §  (t)  represents  a  change  in  a  voltage  or  of  a  current,  regarded  aa  a  station¬ 
ary  random  process,  then,  as  can  be  seen  from  (5.21),  the  quantity  ”  8(0)  is  equal 
to  the  average  power  developed  by  the  process  at  a  load  of  1  ohm. 

The  limit  Jn  the  right  part  of  (5.21)  yields  the  square  of  the  effective  value 
of  the  voltage  and,  consequently,  establishes  the  equivalence  of  the  square  of  the 
effective  value  to  the  quantity  B(0)  (or  to  the  dispersion  of  the  random  process,  if 
its  mean  value  is  equal  to  zero). 

The  ergodic  theorem  is  not  only  of  theoretical,  but  also  of  considerable  practi¬ 
cal  interest.  From  the  equivalence  of  the  two  methods  of  averaging  stationary  random 
processes,  it  follows  that  within  the  framework  of  the  correlational  theory  there  is 
no  necessity  of  studt^ing  a  large  aggregate  of  realizations,  which  the  researcher  as 
a  rule  does  not  have  at  his  disposal,  but  there  suffices  one  single  realization, 
observable  over  an  extended  period  of  time.  Thus,  for  example,  the  statistical  proper¬ 
ties  of  fluctuation  noises,  which  constitute  a  stationary  random  process,  are  studied 
in  the  course  of  a  sufficiently  long  time  on  one  radio  receiver,  and  then  the  resi'lts 
of  this  investigation  may  be  extended  to  all  Identical  sets.  Since  actual  ob¬ 
servations  are  limited  in  time,  the  equality  of  the  aggregate  averages  with  tine 
averages  nay  be  accepted  with  but  a  definite  degree  of  certainty  and  precision.  This 
circumstance  is  emphasized  in  Table  6  in  which  the  formulas  of  the  two  methods  for 
averaging  random  variables  and  stationary  random  processes  are  compared. 


F-TS-98II/V 


173 


m. 


m- 


Table  6 


Random  variable 

Stationary  random  process 

Average  of  aggregate  of  possible 
values 

Average  of  aggregate  of  realleations 

«0 

m 

a  »  m,  {£(/)}  =  J  xwi  (x)  dx. 

J  x»(x)dx' 

—  « 

fiW  =  mi{£(05(/  +  t)} 

00  «Q 

—  00  —CO 

1 

Arithmetic  mean  of  independent 
observations 

Time  average  of  one  realization 

.  Si  +  Ci*t--‘4-c« 

*  n 

»,(r)=yj5‘'>(0rf/ 

0 

.  r 

law  of  large  numbers 

Ergodic  theorem 

llmP{|5-o|<»}=i 

llm  /’{lf,(r)-a|<.)  =  1. 
r  00 

Urn  P{|*,  (r,t)-B(T)|<»}  =.  I 
r  -♦oo 

For  practical  calculations  of  average  values  and  of  correlation  functions  of 
stationary  ranoom  processes  from  experimental  curves  of  ?  (t)  it  is  necessary  in 
formulas  (5*19)  and  (5.20)  to  replace  the  integrals  by  the  integral  sums 


*•1 


and 


*•1 
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nherc  8  is  sufficiently  small,  so  that  over  the  period  of  tlmeS  it  would  not  be 


*  '  / 

necessary  to  allow  for  changes  in  the  curve  of  ^  (t),  and  N  is  selected  to  make 

so  great  that  an  additional  increase  in  the  number  of  items  in  the  sum  would 


have  little  effect  on  the  result  of  the  suamatlcn.  In  recent  years  special  calcu¬ 


lating  machines  —  correlators  —  have  appeared,  which  considerably  facilitate  the 


laborious  calculation  of  correlation  f\inctlon  from  experimental  curves. 


The  basic  elements  of  every  correlator  are  the  delay  circuit,  the  multiplier. 


the  Integrating  circuit,  and  the  recording  apparatus.  Depending  whether  the  multi¬ 


plication  is  performed  by  the  digitol  or  analogue  methods,  the  correlators  are 


classified  (similarly  to  computers)  as  digitol  or  analogue  types.  The  former  are 


more  precise,  but  are  considerably  more  complex  in  design.  Tne  latter  are  simpler 


and  in  many  practical  cases  provide  s.  *  '.factory  precision.  Integration  (accumu¬ 
lation)  may  be  accomplished,  as  is  well  known,  by  means  of  a  simple  RC  circuit.  A 


more  rigorous  examination  of  the  integration  process  in  the  RC  integrator,  especially 


necessary  in  the  investigation  of  short-term  correlation,  makes  it  necessary  to  intro¬ 


duce  under  the  sign  of  the  correlation  Integral  still  another  function,  which  takes 


into  account  the  characteristic  of  the  RC  circuit,  i.e.,  makes  it  necessary  to  re¬ 


place  the  ordinary  correlation  function  by  the  so-called  short-term  auto-correlation 


function  [l  6] 


¥ 

B  (x/  /)  =  2a  J  5  (X) ;  (X  +  X)  e-^‘'->dx. 


whereat  =  5^ 
2RC 


Let  us  note  that  In  the  ergodlc  theorem  the  stationarity  of  a  process  is  under¬ 


stood  in  the  precise  sense.  In  order  to  emphasize  the  importance  of  this  re.-nark,  let 


us  examine,  for  instance,  the  determined  periodic  process  S(t)  with  a  pieriod  of  T. 


Since  the  aggregate  of  the  realizations  of  the  process  consists  of  the  one  function 


S(t),  the  average  for  the  aggregate  coincides  with  this  function,  i.e.,  depends  on 


time,  idiile  the  average  over  time  is  a  direct  component  of  the  process.  Thus  the 


average  for  the  aggregate  does  not  coincide  in  this  case  with  the  time  average.  It 
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C<  •  -■  ^^vft.-'l.  ”*  ‘  ^',,  . 


is  not  difficult,  however,  to  show  that  the  autocorrelation  function  of  a  periodic 
process  depends  only  on  the  temporal  displacement  T=  t2  -  t^.  In  actual  fact, 
representing  the  periodic  function  ly  the  Fourier  series 

S«)=j«,cos(^(  +  ,V 
<1-0  '  ' 

we  obtain  (by  virtue  of  the  periodicity,  the  average  Is  taken  for  a  period) 

®  W  =  -f  J  |]  J  COS  1  cos  1?^  {t 

But  in  view  of  the  orthogonality  of  the  trigonometric  fwetions,  the  integrals  of  the 
product  of  the  cosines  when  n  ^  i.  are  equal  to  zero,  and  when  n  =  k  ^  0 

^  J  cos  (~  t  +  cos  j?^  (/  4-  -)  4-  j  ^  cos  ^  T. 


Thus,  for  a  periodic  process,  the  autocorrelation  function  is 


(5.22) 


where S  =  2,  when  n  =  0,  and 8^  =1,  when  n  >  1, 

It  follows  from  (5.22)  that  the  autocorrelation  function  of  a  periodic  process 
is  a  periodic  function  with  the  period  of  the  process  in  question,  the  amplitudes  of 
the  harmonics  of  the  fundamental  frequency  being  obtained  from  the  corresponding 
amplitude  of  the  harmonic  of  the  periodic  process  by  means  of  squaring  and  dividing 
by  two.  The  autocorrelation  function  does  not  depend  at  all  on  the  phase  angles 
which  contain  harmonics  of  the  initial  perjodic  process. 

4,  Properties  of  the  Correlation  Function 
Let  us  examine  first  of  all  the  behavior  of  a  correlation  function  with  an  un¬ 
limited  increase  of  the  argument  Z  »  If  a  random  function  does  not  contain  dctemlned 
(periodic)  components  (a  purely  random  process),  the  dependence  between  the  variables 
%(’'t-*‘Z)  and  «'>4't  weaken  idien  Z-^  oo  and  these  variables  become  Independent 
at  the  limit.  Since  the  mean  value  of  a  product  of  Independent  random  variables  is 
equal  to  the  product  of  the  mean  values  of  the  comultiples,  and  since  for  a  statlon- 
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Thus,  the  dispersion  of  a  stationary  random  process  is  equal  to  the  difference 
of  the  Talues  of  the  coirrelation  function  when  T  =  0  andT  =  oo,  i,e.,  to  the  dif¬ 
ference  between  the  average  power  of  the  process  and  the  power  of  the  direct  com¬ 
ponent. 

to  view  of  the  stationarity,  i,e,,  of  the  independence  of  the  distribution 
functions  from  the  origin  of  the  time  reckoning,  the  correlation  function  B(T)  is  an 
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the;*efore 


(5.27) 


Ihus,  no  value  of  the  correlation  function  can  exceed  the  value  of  this  function 


iihen  T  =  0, 


This  affirmation  follows  also  from  (5.19)  and  (5.21),  since 


j  (0  (/ + 1)  d/  <  j 


Figure  37  shows  a  typical  curve  of  the  correlation  function  of  a  random  process, 

iMch  illustrates  the  above  enumerated  properties  of  this  function.  It  should  how- 

ever,  be  noted  that  the  asymptotic  approach  of  B(r)  to  the  quantity  a  idienf — >  oo 

does  not  always  take  place  monotonously;  there  may  be  cases  when  the  values  of  the 

2 

correlation  function  fluctuate  about  a  ,  approaching  this  quantity  with  an  increase 
of  r  , 


^  Bfihtfiuattptd- . 


Legend:  ’ 


- ~  a)  B(o)-general  mean  power 

J - j - -  b)  B(oo)-pow3r  of  the  constant 

*  co)i?)onent 

Fig.  37.  laical  curve  of  correlation  function  cf  a 
purely  random  stationajy  process. 

Let  us  remarK  that  equality  (5.19)  established  for  stationary  random  processes 

the  actual  physical  sense  of  the  correlation  function  as  a  function  which  character- 

ites  the  connection  between  the  preceding  and  succeeding  values  of  ^  (t).  As  a 

measure  of  this  connection  we  shall  take  the  amount  2E(rj  of  the  mean-square  deviation 
(k) 

of  ^  (t)  from  this  same  realization,  but  shifted  by  an  interval  r  along  the  time 


axis  (Fig.  3d}»  l.*.t 


2.  (t)  =  ^im  4-|  (0  -  6“’  {t + 
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Fig.  38*  Time  shift  of  the  realization 
of  a  random  process. 


Squaring  the  difference  of  the  functions,  breaking  the  Integral  of  the  sums  into 


a  sun  of  integrals  and  employing  the  ergodic  theorem,  we  obtain 


W + •  W = B  (0) = const. 


Consequently,  the  correlation  function  B(t^)  ■  supplements  the  magnitude  of  the  mean 


Ck)  M 

square  deviation,  ^  (i  z)  ,  from  ^  »  to  a  constant  equal  to  the  power  of 


the  stationaiy  randoa  process  (Fig.  39),  and  indicates  in  this  manner  the  extent  to 


which,  on  the  average  two  values  of  a  random  process,  separated  by  a  time  interval  of 


r  are  linked  to  each  other. 


Fig.  39»  link  between  £  (r)  and  B  (z)» 


In  place  of  a  random  process,  there  are  frequently  examined  the  deviations  from 


its  mean  value  called  the  pulsations  of  a  random  process,  or  fluctuations* 


♦  Sometimes  a  fluctuation  is  called  the  numerical  characteristic  of  a  random  variable 
-  the  square  root  of  the  dispersion  (cf.  p.  65).  It  sJwuld  always  be  kept  in  mind, 
in  which  of  these  two  completely  different  senses  this  term  is  employed. 
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The  correlation  function  B^(r)  of  the  pulsations  of  a  stationary  randon  process 
is  equal  to 

a,W=m,  {eo(/)5o(/-t-t)}=m,  {l(05(/  +  t) -a5(0- 

— (5.28) 

and  since  the  nean  value  of  the  sum  is  equal  to  the  sum  of  the  mean  values*  therefore 

^(t)=B(t)— 0*.  (5.29) 

It  follows  from  (5.29)  that  the  mean  value  of  the  pulsation  is  equal  to  *ero, 
and  its  dispersion  to 

>  a*=B,(0).  (5.30) 

The  ratio  (5-31) 

> 

is  called  the  coefficient  of  correlation  of  a  stationiiry  random  process  (or  of  a 
stationary  random  pI^^cess  with  a  zero  mean  value).  The  quantity  R(t)  is  some  tines 
called  the  r.ormall2€d  correlation  function.  This  tern  is  more  precise,  but  the  tern 
•coefficient  of  -  correlation"  has  become  entrenched  in  the  literature. 

Comparing  (5*31)  with  (2.91),  we  conclude  that  R(r)  is  the  coefficient  of  cor¬ 
relation  of  the  two  random  variables  ^(t^)  and  ^ (tg),  with  tg  =  t^  ♦  . 

The  coefficient  of  correlation  R(r)  has  the  same  properties  as  the  correlation 
function.  It  constitutes  an  even  function  of  this  argument.  The  naxiaua  value  cor¬ 
responds  to  f  =  0  and,  according  to  (5.27)  and  (5*31).  R(0'  =  1,  and  jR(T)|^1  for 
ary  t ,  For  a  purely  randon  process  when  t  -»  cxi,  R  (f )  -?■  0,  The  coefficient  of 
correlation  may  assune  values  of  zero  even  for  a  finite  value  of  Tr  ,  KoiTever,  the 
equality  of  the  coefficient  of  the  correlation  does  not  yet  signify  independence, 
whereas  two  independent  random  variables  are  always  uncoordinated,  i.e,,  for  them 
always  R  a  0  (cf*  p«  74  ), 

For  a  purely  random  process,  .t  Is  alw&ys  possible  to  indicate  such  a  ,  that 
when  ■*!'  >  T^,  the  variables  ^(t)  and  ^(#+t)  may  be  considered  practically  inde¬ 
pendent,  practical  iivdependence  being  understood  in  the  sense  that  when  '>'^q  *  the 
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absolute  anount  of  the  coefficient  of  correlation  remains  smaller  than  a  given 
quantity,  for  instance 

|/?W|<0.05.  , 


(5.31)  /iici 

•1 

The  quantity  t  is  called  the  time  or  the  interval  of  correlation.  i 

■  ^ 

Sometimes  the  time  of  correlation  tn  is  defined  as'  half  the  iridth  bf  the  base  of  i 

I 

,g 

a  rectangle  of  unit  height,  whose  area  is  equal  to  the  area  under  the  correlation 

--t| 

coefficient  curve,  i.e. ,  i 


— 0 


(5.32) 


In  the  future,  when  not  the  correlation  function  of  a  process,  but  the  coef-  | 
ficient  of  correlation  is  discussed,  it  is  necessary  to  remember  that  the  latter 

refers  to  the  pulsation  of  a  process  or  to  a  process  with  a  zero  mean  value,  | 

5.  Detection  of  a  Periodic  Signal  in  Noise  | 

The  so-called  correlation  method  of  receiving  a  periodic  signal  concealed  in  | 
noise  is  based  on  the  employment  ^'f  the  properties  of  the  correlation  fimctlon.  § 

Let  ^ (t)  be  the  stationary  random  process  representing  noise,  and  S(t)  the  | 

periodic  signal.  Let  us  examine  the  autocor-^lation  function  L(t/  of  the  sum  of^(t)  | 
and  S(t),  In  accordance  with  (5*19)  we  have  .1 


B  (t) -f-jm+s  (01  mt+x)+s(t-}--.)]dt= 

with  the  designations  of 


(5.33) 


(t)  =  j  5  (/)  5  ^ 

0 

(5.34) 

r 

<') J  ^  (0  5  (/  +  T)  d(, 

0 

(5.35) 

r 

(') = f  1  (/)  s  {/  ^  t)  dt. 

(5.36) 
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Ws!  *  A-y  T»  ‘VJSs,.'  U1 »'  i-i?-  •';• 


fl,«  ^)= g^T  p  (0  5  (/ +  ^) «//. 


(5.37) 


The  functions  B^(r)  and  B^(f)  represent  respectively  the  autocorrelation 

functions  of  noise  and  signal.  The  functions  B. and  B  are  called  Bjutual  cor- 

SiC  cju 

relation  function  of  the  processes  $ (t)  and  S(t),  If  the  processes  ^ (t)  and  S(t) 

are  independent  and  the  mean  value  of  one  of  these  proccssos  is  equal  to  sere,  then 

B  pO, 

luc  CiT 

Processes  for  tdiich  the  mutual  correlation  functions  are  equal  to  a  constant  or 
turn  to  *ero  are  often  called  incoherent  functions.  In  the  general  case,  mutual  cor¬ 
relation  functions  can  serve  as  a  measure  of  the  link  (coherence)  between  the  two 
processes. 

Since  the  signal  and  the  noise  at  the  input  of  a  receiving  set  m^,  as  a  rule, 
be  considered  independent,  the  mutual  correlation  functions  in  (5»33)  disappear  and 


WM 


(5.38) 


i.e, ,  the  autocorrelation  function  of  the  sum  of  the  noise  and  the  periodic  signal  is 
equal  to  the  sum  of  the  autocorrelation  functions  of  the  noise  and  of  the  signal.  As 
has  been  shown  above,  tlie  correlation  function  of  a  random  stationary  process,  such 
as  the  noise  is,  coincides  with  its  autocorrelation  function  and  diminishes  with  an 
unlimited  increase  of  the  argument.  For  a  periodic  signal  the  autocorrelation 
function  is  periodic,  with  a  period  equal  to  the  period  of  the  signal.  Therefore,  ty 
switching  a  correlator  into  the  receiving  set,  it  is  possible  after  some  time  to  de¬ 
tect  whether  the  correlation  function  of  the  received  signal  has  a  periodic  component. 
If  such  a  periodic  component  stands  out  when  T  is  increased,  this  will  indicate  the 
presence  of  a  signal;  if  however  with  the  growth  of  r  the  value  of  the  correlation 
function  tends  toward  a  constant  value  (in  particular  towards  tero)  then,  consequently, 
a  signal  is  lacking*. 

•  For  short  periods  of  the  observation  time  T,  all  four  terms  of  equation  (5.33)  will 
influence  the  findings  of  the  correlator,  but,  with  a  sufficiently  large  tine  of 
integration,  the  terns  corresponding  to  the  mutual  correlation  of  signal  and  noise 
will  disappear, 
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Figure  40  showij  the  autocorrelation  functions  of  noise,  of  a  periodic  signal, 
and  of  their  su«.  The  autocorrelation  function  of  the  sum  has  an  absolute  maximun 
vhen  7  s  0  and  relative  maximuns  which,  starting  with  some  '2'  ,  are  repeated  with  a 
period  practically  equal  to  the  period  of  the  signal.  If  P  is  the  power  of  the 

V 

signal,  tod  P  itt  the  power  of  the  noises,  then  from  (5.2l)  and  (5.22)  it  follows  that 
v  • 

=  +  B{nT)=P^  ■ 

and,  consequently,  the  relative  difference  between  the  absolute  maxinun  and  the 

periodically  repeating  naximuns  of  the  autocorrelation  function 

B(0)-S(nT)  Pm 
B{nT)  —  /». 

Is  smaller,  the  more  the  power  of  the  signal  exceeds  the  power  of  the  noise.  There* 
fore,  with  a  strong  signal,  the  detection  of  a  periodic  signal  in  noise  nay  be  ac¬ 
complished  sufficiently  rapidly,  whereas  with  a  weak  signal,  in  order  to  obtain  the 
correlation  function  of  a  large  argument  r  ,  tine  will  be  required  for  the  retention 
of  the  process  for  tine  r  ,  and  for  the  averaging  over  the  time  of  nT  »  r.  Thus 
although  theoretically  no  limitations  exist  for  the  detection  of  weak  signals  by  the 
correlation  method,  in  practice  there  will  exist  threshold  values  for  signals “de¬ 
tectable  in  noise,  determined  by  a  finite  observation  time  and  ly  the  solving  capacity 
of  the  instruments. 


•ii  'I'lii'f  I  Ti  II  "I  I  '~i  ~  I II I  ill . faiis 
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Pig.  Correlation  fUnctionsj  a)  of  noise;  b)  of  periodic 
signal;  c)  of  sum  of  periodic  signal  and  noise. 


In  cases  vhen  it  is  necessary  to  detect  a  periodic  signal  S(t),  the  form  of 


uhieh  is  known  in  advance,  the  correlation  method  of  separating  the  signal  from  noise 


may  be  improved  on  the  basis  of  an  a  priori  knowledge  of  the  function  s(t) (coherent 
reception).  For  this,  the  signal  S(t)  is  reproduced  at  the  receiving  end  by  a  local 


oscillator,  and  there  is  studied  the  mutual  correlation  function  of  the  incoming 


signal,  distorted  by  noise,  and  of  the  signal  from  the  local  oscillator,  i.e.. 


r 

B  (t) = lim  ^  C 1;  (/)  -1-  S  (/)]  S  (/  + 1)  d/  =  (,)  +  (t). 


Since  ^  it)  and  S(t)  are  independent,  therefore,  as  has  already  been  poiiited  out  above 


B it)  *  0  and,  consequently. 


Bi^)=B^(x). 


(5.38») 


Thus,  in  the  case  under  consideration  the  autocorrelation  function  of  the  signal  is 
directly  obtained  at  the  output  of  the  correlator,  and  no  supplementary  time  is 
needed  for  the  removal  of  the  aperiodic  item  in  (5*3^)  which  is  caused  by  noise. 
Clearly  in  this  case  too,  the  remark  made  on  page  182  ,  concerning  the  influence  of 


the  term  B  (r)  since  the  averaging  time  T  is  finite,  remains  in  force. 

Vf  V 


Besides  those  indicated  above,  such  methods  of  receiving  weak  signals  as  accumu¬ 


lation  and  filtration  are  at  present  known  and  in  widespread  use.  The  essence  of  all 


these  methods  is  described  mathematically  by  the  averaging  operation 
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The  function  f(t)  under  the  integral  sign  determines  the  method  of  reception.  Thus, 
with  the  autocorrelation  method,  f(t)  =  $  (t)  +  S(t),  and  with  the  coherent  method, 
f(t)  =  S(t),  Accumulation  corresponds  with  f(t)  s  1,  and  filtration  with  f(t)  =  h(t), 
where  h(t)  is  the  pulse  transfer  function  of  the  filter  (cf.  Section  1,  Ch.Vl), 

6,  Power  Spectrum  of  a  Random  Process 

Bie  extent  to  which  harmonic  analysis  has  been  fruitfully  employed  in  the  studjy 
of  non- random  (determined)  processes  is  well  known;  Fourier  series  for  periodic,  and 
the  Fourier  integral  for  aperiodic  processes.  It  would  be  desirable  to  have  such  a 
simple  and  effective  n^thematical  apparatus  for  the  stu^y  of  random  processes.  The 
direct  application  of  classical  harmonic  analysis  to  random  processes  is  impossible, 
since  analytically  these  processes  are  not  expressed  as  precise  functional  relation¬ 
ships  of  an  independent  variable.  It  is,  however,  possible  to  generalize  harmonic 
analysis,  averaging  the  spectral  breakdowns  obtained  from  individual  realizations, 
with  respect  to  time, 

'i) 

With  this  purpose,  let  us  examine  one  realization  ^ (t)  of  a  random  process, 
(k) 

Let  in  addition  ^  ^(t)  be  a  function  equal  to  zero  outside  the  interval  of|tJ  ^T/2 

^  (k) 

and  coinciding  with  ^  (t)  within  this  interval.  The  spectrum  (Fourier  transfor- 

(kl 

matlon)  of  function  $  ,'(t)  has  the  form  of 


(5.39) 


However,  for  the  function  ^  '"*(t)  bounded  in  time  by  the  interval  of |t|  ^  T/2, 

T 

the  continuous  spectrum  is  uniquely  determined  by  the  assignment  of  spectral 

2%k  aff 

intensities  at  the  discrete  points  «  »  —  ,  located  at  a  distance  of  “j"  from 

each  other.  Actually,  resolving  the  function  5^^^(t)  into  a  Fourier  series  over  the 
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interval  of  -T/2  <  t  <  T/2,  we  obtain 


•  itiit. 


irtiere 


r 

r  _*:i5, 
r  fC«)«  '  <“■■ 


.yuf  Ji 


1  m  /2l{//t\ 


(5.'<o) 


(5.40*) 


Substituting  (5*^)  into  (5«39)  and  cnploying  (5.^*)*  after  the  simplest  computations 
we  then  obtain 


-J"  —  *ft 


Relationship  (5.^1)  yields  the  indicated  link  between  the  continuous  3pecti*um  Z^^(iu)) 
and  its  values  at  the  discrete  points  of 

If  §  ^  is  a  voltage  or  current  at  a  load  of  1  ohm,  then  the  mean  power  of  a 
process  at  a  frequency  of  3^^,  with  respect  to  the  frequency  band  of  Af  =  i/t,  will 
be  equal  in  accordance  with  (5*^)  and  (5.^’)  to 

Passing  to  the  limit  when  T— j^co,  we  obtain 

■  C*(«)=Iimf  lZ*r(/u.)l*. 

r-»oo  ' 

which  has  the  dimensions  of  power  per  unit  frequency  band,  and  represents  the  spectrum 

(k) 

density  of  the  mean  power  of  process  ^  '(t). 

The  spectrum  density  Gj^Cw)  can  exist  even  when  a  Fourier  transformation  of 
(k) 

$  ^(t)does  not  exist,  i.e,,  when  lim  Zj^,^iw)  =  co  .  The  introduction,  in  place  of 
the  usual  spectrum  characteristic  Zj^^(iw),  of  the  spectrum  density  Gj^Ctt)  is  just  that 

•  The  property  of  the  spectrum  of  a  function  limited  in  time,  expressed  by 
is  analogous  to  the  property  of  a  function  with  a  limited  spectrum,  which  comprises 
the  content  of  the  well-known  theorem  of  V.  A.  Kotel’nikov,  according  to  which  the 
function  f(t),  having  a  limited  spectrum,  is  completely  determined  by  its  discrete 
values  at  points  situated  at  a  distance  of  l/2F  wi.th  respect  to  one  another,  where 
P  is  the  maximum  frequency  in  the  spectrum  of  f(t) 
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generalization  of  harmonic  analysis  which  makes  it  possible  to  expend  the  usual 

spectrum  concepts  to  functions  which  do  not  satisfy  the  condition  of  absolute 

integrability  (i.e.,  those  for  which  a  Fourier  integral  does  not  exist).  An  ex- 

(k) 

ample  of  such  functions  are  the  '(t)  realizations,  under  discussion  of  a  random 
process. 


Let  us  show  that  the  spectrum  (Fourier  transformation),  of  the  product  of 

kT^ 


(k)  (k)  9 

£  (t)  ^  (t  +r)  averaged  over  time,  coincides  with  2/tI  Z,  -(iw)  1 

T  T 


Truly, 


2][t  +  = 


f  J  ^*’(0  e'-'rf/  J  +  .)  = 


r 

1 


'  r  Jr 

T  2* 

from  which,  taking  into  account  (5* 39).  we  obtain 


=  -f  (-  iu.)  Z„  (i«.)  =  -f  1  (/u,)  1* 


Passing  to  the  limit  when  T— ♦«»  ,  we  find  the  spectrum  density  Gjf(^*^)  of  the 


(k). 


mean  power  of  realization  §  (t) 


0^  («*)  =  lira  Y I  ^*r  M  I* = 

r-Mo  *  ' 


(5.^2) 


1  . 


R)ra  stationary  random  process,  the  limit  enclosed  by  the  curved  brackets  does 


(k), 


not  depend  on  the  choice  of  the  realization  §  ^(t)  of  a  random  process  and  is  equal 


to  the  correlation  function  of  the  process  (cf.  Section  3), 
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thus,  for  a  stationary  random  process,  the  spectrim  density  of  the  mean 
power  of  the  process  ^(t)  is  the  sane  for  any  realization  of  the  random  process 
and  is  equal  to  the  spectrum  ?(tO)  of  the  correlation  function  B(r) 


09 

c*  H = Urn I  (im)  P = 2  J  B  (t)  e-'- Vt = f  (a.). 


(5.42*) 


This  spectrum  density  of  the  mean  power  of  any  realization  of  a  process,  coin¬ 
ciding  with  the  spectrtun  F(<^)  (Fourier  transformation)  of  correlation  function  B  (X), 


is  called  the 


of  a  stationary  random  process.  This  spectrum’  provides 


only  an  a’\rei‘age  power  picture,  the  distribution  of  the  power  of  a  process  among  the 
frequencies  of  the  elementary  harmonic  conponents,  but  does  not  account  for  the  in¬ 
stantaneous  phases  of  these  components. 

As  has  been  shown  by  A.  Ya,  Khinchin,  the  correlation  function  of  a  stationary 
random  process  may  always  be  presented  in  the  form  of  an  integral 


0»  ,  ^ 

B  (t)  =  J  f  (u»)  e^*”da)  = 


(5.43) 


where  F(w)  is  a  limited  function  (F(a>)>’  0),  Conversely,  every  function  B('r),  which 
may  be  presented  in  the  fora  of  (5.43),  is  the  correlation  function  of  some  statlonaiy 
randjffl  process,  Ihe  function  F(<J)  is  the  direct  Fourier  transformation  of  the  cor¬ 


relation  function; 


m  eo 

f  («,)  =  2  j  B  (t)  e~‘*’dx = 4  f  B  (t)  cos  Wt. 

— flO  • 


(5.44) 


This  function  has,  as  has  been  shown  above,  a  clear  physical  sense:  it  represents 
the  spectrum  density  of  the  mean  power  of  the  process. 

Let  us  remark  that  in  foTOula  (5*i*3)>  'frith  employment  of  the  Fourier  transforma¬ 
tion  in  an  exponential  form,  the  concept  of  the  spectrum  distri^'^ution  of  the  mean  power 
of  a  process  was  extended  over  all  real  frequencies  from  dJs:  -cdto  CO»-vcp,  Only  the 

li 

positive  frequencies  of  from  u)  ~  0  toWs+co  make  physical  sense.  To  make  pos¬ 
sible  the  employment  of  the  ejqponentlal  fom  of  the  Fourier  integral,  which  simplifies 
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its  cc.-*putation,  each  spectruai  component  is  broken  down  into  the  two  equally  in¬ 
tensive  components  F(w)  and  i  F(-«)  in  such  a  manner  that  the  total  power  spectrum 
F(w),  distributed  over  the  negative  frequencies,  becomes  an  even  function  of  the 
frequenqr  (idiich  can  also  be  seen  from  if  it  is  remembered  that  the  cor¬ 

relation  function  is  always  even).  It  should,  however,  always  be  remembered  that 
the  property  of  the  evenness  of  a  power  spectrum  is  preserved  only  if  the  origin  of 
the  coordinates  lies  at  the  point  of  w  =  o,  and  may  be  violated  if  the  origin  is  dis¬ 
placed  to  another  point. 

Since  a  Fourier  transformation  is  possible  only  under  the  condition  of  the  abso¬ 
lute  integratibility  of  a  function,  therefore  formulas  (5.43)  and  (5.44)  are  valid 


under  the  condition  that 


JlB(t)|dt<M,  j’f(u))do.</V, 


(5.45) 


where  K  and  N  are  constants. 

Condition  o.45)  restricts  the  applicability  of  fonaulas  (5.43)  and  (5.44)  only 
for  stationaiy  processes,  the  mean  value  of  which  is  equal  to  zero  (no  constant  com¬ 
ponent),  and  which  also  have  no  periodic  components.  In  the  work  of  Khinchin  [2] 
the  property  in  question  of  stationary  random  processes  is  formulated  in  a  more 
general  fora,  so  that  the  limitations  of  (5.45)  are  insignificant.  Eelow  it  will  be 
shown  how  formulas  (5*43)  and  (5»44)  are  generalized  for  a  stationary  process,  con¬ 
taining  both  a  constant  and  periodic  coii^onbnt* 

From  (5.43)  when  =  0  we  find  that  the  mean  power  of  a  stationary  process  is 


equal  to 


B(0)= 


(5.46) 


i.e..  Is  equal  to  the  area  of  its  power  spectrum.  The  spectral  density  of  the  mean 
power  when  6*=  0,  as  can  be  seen  from  (5.44),  is  equal  to 


f(0)=2  jB(t)dT. 


(5.47) 
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I  i 


;<5 


i.e.,  to  double  the  area  under  the  curve  of  the  correlation  function.  The  ratio 


.  fJP] _  may,  in  accordance  with  (5»32).  be  taken  as  a  measure  of  the  time  of  cor- 

4  6(0) 


■5? 


relation. 

The  correlation  function  BCtr)  and  the  pov;er  spectrum  F(<a))  of  a  stationary  random 
process,  as  a  pair  of  Fourier  transformations,  possess  all  the  properties  intrinsic 

3 

V'- 

to  this  transformation.  Specifically,  the  "wider"  is  the  spectrum  F(a)),  the  "narrower"  » 
is  the  correlation  function  B(i;),  and  Tice  vex'sa.  In  other  words,  the  product  of  the 
correlation  time  T:q  and  bandwidth  of  the  porj-er  spectrum  is  a  constant. 


Fbr  a  nonstationary  random  process,  the  spectrum  density  ^or  various 


realisations  $  ^^^(t)  of  a  random  process  varies.  Therefore  for  a  nonstationaiy 
random  process  the  power  spectrum  of  the  process  Is  called  the  average  of  an  aggre¬ 
gate  of  Gjj(w),  i.e.,  the  quantity 

m,  {C7,  H j  =m,  I  to  ^  1  Z„  (/u.)  pj  = 


=  to-fm,{lZ,rMPl. 


(5.48) 


.  It  follows  from  (5.48)  that  the  power  spectrum  of  a  nonstationary  random  process 
coincides  with  the  spectrum  (Iburier  transformation)  averaged  for  the  time  of  the 
correlation  function  of  this  process 


dT= 


(5.49) 
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Formula  (5.49)  is  a  generalisation  of  (5.42*)  for  nonstationary  random  processes. 

Fk*om  tdiat  has  been  stated  above,  it  follows  that  two  ways  exist  for  computing 
the  power  spectrum  of  a  random  process: 

(k) 

a)  direct,  proceeding  from  the  observation  of  a  single  realization  $  (t)  and 

then  finding  the  limit,  lim  2/T  i  Z  (i«)  J  or  the  average  of  an  aggregate  of  this 
limit  for  a  nonstationary  random  process; 

b)  finding  the  Fourier  transformation  of  a  correlation  function  for  a  stationary 


random  process,  or  a  correlation  function  averaged  over  time  for  a  nonstationary 


7«  Power  Spectrum  of  Generalised  Telegraph  Signal 

Let  US,  as  an  example,  compute  the  correlation  function  and  power  spectrum  for 
the  generalized  signal  $  (t)*,  one  of  the  realizations  of  idiich  appears  in  Figure  2 
(p*  34  ).  It  can  be  shown  that  the  probability  of  exactly  k  sign  changes  taking 
place  in  the  signal,  during  the  time  t,  is  determined  by  the  formula  of  Foisson 


(1«5^)  (cf#  Section  9>  Ch.  1) 


upon  the  fulfillment  of  the  following  condition:  no  matter  how  astny  sign  changes 
take  place  in  the  time  period  (O,  t),  the  probability  of  one  sign  change  over  the 
interval  of  (t,  t  +At)  is  equal  to  MAt,  and  the  probability  of  more  than  one  sign 
change  decreases  more  rapidly  than  At.  Here  >>  is  the  constant  average  number  of 
sign  changes  in  a  unit  of  time,  and  Pt  is  the  average  numoer  of  sign  changes  during 
the  time  t.  In  the  general  case,  the  mean  density  of  the  points  of  intersection  of 
the  curve  with  the  abscissa  (cf.  Fig.  2)  may  be  a  function  of  tine. 

Designating  this  function  by  ^  (t),  we  find  the  average  nxuaber  of  sign  changes 


over  the  tine  interval  of  from  t^  to  t«, 

1  2 


X=|v(0d/. 


(5.50) 


Since  (t)  ^  0,  therefore  A.>0. 


The  probability  that,  over  the  tine  interval  from  t^  to  t^  exactly  k  sign  changes 
of  the  signal  will  take  place,  will  be  equal  to 


(5.51) 


*  The  generalized  telegraph  signal  is  a  special  form  of  the  so-called  Poisson  random 
process,  a  detailed  investigation  of  which  appears  in  [5]  . 
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Formula  (1.5^^)  is  a  special  case  of  (5.50.  when  y(t)  is  a  positive  constant. 

To  compute  the  correlation  nmetion  of  a  generallaed  telegraph  signal  it  is 
necessary,  according  to  definition  (5.16).  to  obtain  an  average  of  the  aggregate 
realiaations  of  the  product  ot  $(tj)  This  product  nay  be  equal  either  to 

Qj.  ^  j,2  pj^g^  2)  depending  on  whether  ^  (t^)  =  $(12)  or  ^  (t^)  =  -  ^(t^). 
The  equality  ^(t^)  =  signifies  that,  over  the  interval  from  t^  to  t2.  an  even 

number  of  sign  changes  occurred,  i.e..  there  occurred  one  of  the  mutually  incompaU- 


ble  events:  k  =  0.  ork  =  2,  ork=4.,,. 


•Ihen.  employing  (5.51).  we  find  by  the  rule  of  addition 


m  * 

P  lE(/,)=5(/2)i  =  chi. 


(5.52) 


Analogously,  the  equality  $(t,)  =  -  signifies  that,  over  U^e  interval 

of  from  t^  to  t2.  there  occurred  an  odd  number  of  sign  chari.»^es  and 

fc 


^iU(2a-i)! 


(5.53) 


Nov  it  is  not  difficult  to  obtain  an  average  of  the  product,  i.e,,  the  cor¬ 


relation  function 


B(/,.  /J  =  m,{=(/,);(/,)}  =  A»P{;(/,)=5(/J}- 


(5.54) 


Substituting  (5.52)  and  (5*53)  into  (5.54),  ve  obtain 


/,)  =  A*e~‘ (chi  —  shi)=A»e  ^ 


or,  taking  into  account  (5.50). 


u 


B{ti,  /a=A»e 


—  ^ 


(5.55) 


If  the  generaliied  telegraph  signal  is  stationary,  then  =  const,  and  the 


correlation  function  depends  only  of' the  diff ermco’ 't  •  tg  -  t 


(5.56) 
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^  =  j  (^0  +  ''1  cos  »oO  —  V + 

^  [sin  u)q/  (cos  Wjt  —1)4*  cos  u)j/  sin  (o^t] = -f- 
c  4-^2sin^cos((i)o/4-(p). 

Since  X(t,r)  is  a  periodic  function  with  a  period  of  T  =  ,  it  is  sufficient  to 

awerage  the  correlation  function  over  the  tine  T 


B(t)  = 


or  (cf.  (3.37)) 


/k»e~*’»*  f  «*  <•</+») 


(5.59) 


where  Iq(x)  is  a  Bessel  function  of  an  imaginary,  sero-order  argument.  As  before,  an 
absolute  value  of  r  is  taken  in  the  exponent  by  reason  of  the  evenness  of  the  cor¬ 
relation  function.  WhenPjS  0,  formula  (5.59)  turns  into  (5.56). 

Tb  determine  the  power  spectrum  it  is  useful  first  to  expand  the  Bessel  function 
into  a  Fourier  series 


'.(=;«"¥)=  S'.'"'-' 


whose  coefficients  are 


<•*  — C_*  — 2(  *)*(m^)  (n!)»(rt-*l!(rt  +  *)!> 


Bow,  subjecting  (5.5^^)  to  a  Fourier  transformation,  we  find  the  power  spectrum,  aver¬ 
aged  over  time,  of  the  generalized  telegraph  signal  under  examination 


OB  C» 


f(»)=:2A*  Jjc* 


(  '' 


^W==  S 


(5.60) 
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where 


'(«-*««)* +4  -*  • 


From  t  comparison  of  (5*^)  vith  (5>57)t  it  follows  that  in  the  case  at  hand, 
the  power  spectrum  constitutes  a  linear  combination  of  the  same  type  of  curves  as  in 
figure  41  b,  whose  maxima  lie  in  frequencies  of  k 

8»  Continuity  and  Differentiability  of  a  Stationary  Random  Process 
In  the  majority  of  engineering  applications,  the  correlation  function  B(r)  of  a 
stationary  random  process  is  a  continuous  function  of  this  argument  tr  •  It  is  pos> 
sible  to  show  that,  for  continuity  with  any  r  ,  it  is  sufficient  that  a  correlation 
function  be  continuous  at  one  single  point  of  r  *  0  [l}  , 

The  continuity  of  a  correlation  function  provides  also  for  the  continuity  of  the 
random  stationary  process,  which  signifies  that  for  any  moment  of  time  t  and  for  an 


arbitrarily  small  £>  0, 


limP{|l(/+r)~5(0!>2}  =  0. 


The  random  process  $ (t)  is  differentiable  at  the  point  t,  if  there  exists  such  a 
random  variable  ?*(t),  called  the  derivative  of  process  ^(t)  with  respect  to  point 


t,  that 


Thus,  the  statistical  characteristics  of  the  derivative  $*(t)  may  be  obtained  ly 
a  limit  transition  from  the  statistical  characteristics  of  the  difference  of  $(t  +  T) 
—  $(t).  In  certain  problems,  calculation  of  the  statistical  characteristics  of  the 
difference  $(t  +  T)  —  ^(t)  nay  be  independently  significant.  We  shall  restrict 
ourselves  to  determining  the  one-dimensional  distribution  function  of  this  difference 
and  its  correlation  function  and  power  spectrum. 

If  the  two-dimensional  distribution  function  W2(X|,  X2,  T)  of  the  process  ^  (t) 
is  known,  then  employing  (3*24’ we  calculate  the  one-dimensional  distribution 
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function  of  the  difference  of  the  random  variables 

•0 

f  leaCx,,  x+x,,7')rfx,.  (5.6I) 

—  -4.  I 

'  “  \ 

It  is  not  difficult  to  find  the  correlation  function  Bj(r)  of  this  difference, 
if  the  correlation  function  B(r)  of  the  stationary  random  process  ^(t)  is  knoim 


W = «*!  {(« (/ + f)  -  ?  (/) J 15  (/  +  r  + 1)  -  { (/  +  T)1  j  = 

*=«!{?(/+ 7)  E  (/ -f  r -j- 1)|  _/„,{{(/)  5  (/ 4. 7- 4  ,)j_ 

-m,  {?(/-|.7)l(/+t)j4-m,  {5(/)?(/-|-t)J , 

wherefrom 

B,  (t) = 2B  (t)  ~  B  (t  4. 7)  -  B  (t  -  7). 

(5.62) 


the  power  spectrum  Fj(w)  of  the  difference*  is  linked  by  a  simple  relationship  to  the 

power  spectrum  F(w)  of  the  process  ^(t) 

«»  o» 

f,(»)=:2  jBy(t)e“'“"dt=4  J  B(t)e-'*^dt- 

-2  jB(T4-7)c''“’dt-.2  f  B(t-^7)e“'“Vt= 

«0 

=2  J  B(x) - e“  ~  e-'*‘"+”ldx= 

<«0S 

«• 

=  2-2(1  ~ cos u>7;  J  B (t) 


or 


i  7r(«>)=4sin»if.>(a.). 

From  (5.61)  -  (5.63)  it  is  possible  to  find  the  corresponding  statistical  character¬ 
istics  of  the  derivative 

3h  view  of  (3,12)  we  find  that  the  distribution  function  of  Jl.r..  is 

(1) 

equal  to  TWi  (Txi,  T),  and  the  distribution  function  (x)  of  the  derivative  ^’(t) 


(5.63) 


is  obtained  by  taking  the  limit 

<>(x)  =  Iim7Wf^,(7x,.7). 

*  r-.o 

sine.  th.  correUUon'funcUen  ef  i.  .m.!  t”  ft  8,  W  -  there 

fore  from  (5.62)  expanding  the  right  hand  side  into  a  Taylor  series,  we  obtain  for 
the  correlation  function  of  the  derivative  ^'(t)  the  following  expression 
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B.  {-) = lim  ~  Bf  (") = {-  B"  (t)  +  0  (f*) j  =  -  B"  (t). 


(5.64) 


Thus. the  correlation  function  of  the  derivative  ^^(t)  is  equal  to  the  second  deri¬ 
vative  of  the  correlation  function  of  ^(t),  with  the  opposite  sign. 

Tron  (5.53)  by  means  of  taking  the  lisAt,  we  also  find  the  power  spectrum 


of  the  derivative 


f ,  («) = lim  ^  4sia*  y  F  (o)) = u)*f  (w) 


(5.65) 


Formula  (5*65)  follows  also  from  (5*64)  and  (5.43). 

The  dispersion  of  the  derivative  (with  a  sero  mean  value)  is  equal  to 


B,  (0) = — B"  (0) = 5^  f  u>*f  (o))  rfu). 


(5.64*) 


Since  when  r  *  0  the  correlation  function  B(r)  always  attains  a  maximum,  B"(0)<0. 

Let  us  now  determine  the  correlation  function  of  the  differentiable  stationary 
process  ^  (t)  and  of  its  deiivative  $(t). 


Since 


m,  {?  (0 15  (<  +  r  + 1)  -  5  (^  +  T)1 }  =  B  (f  + 1)  -  B  (t). 


the  desired  mutual  correlation  function  is 


«i  {5(05' (<  +  x)} 

r-»o  I  * 


i*®#  t 


T-*0  * 


m,{5(05'(/  +  t)J=B'(t). 


Considering  (5.43).  we  also  find 


(5(05'  Jwf  (a>)sin  (otdu) 


Whenr*  0,  from  (5.66*)  there  follows 

mi{5(05'(0)  =  B'(0)  =  0. 


(5.66) 


(5.66*) 


(5.67) 
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I 


Thus  the  mutual  correlation  function  of  a  statlonaty  random  process  and  of  its 
derivative  In  coincident  time  Instants  is  always  equal  to  zero»  i.e.,  a  random 
function  and  its  derivative  aw»  In  coincident  time  instants,  not  correlated.  From 
this  It  follows  that  the  joint  distribution  function  of  ^  (t)  and  ?*(t)  is  equal 
simply  to  the  product  of  their  one«dlmensional  distribution  functions. 


Wj  (X,  X')  =  tt>,  (x)-  tt)‘,‘’(x'). 


(5.67*) 


There  may  also  be  examined  the  n-th  derivative  of  the  random  process  ICt^ 

for  the  existence  of  -nhich  it  is  sufficient  that  there  exist  a  continuous  derivative 

of  the  2n-th  order  of  the  correlation  function  of  the  process  ^(t).  The  correlation 
function  of  the  n-th  derivative  of  the  process  is  equal  to 


and  its  power  spectrum  to 


),  Random  Processes  with  a  Discrete  Spectrum. 


If  the  conditions  of  (5.^5)  are  fulfilled,  then  the  power  spectrum  FC***)  of  a 
random  process  is  a  continuous,  non-negative  function  of  the  freruency  6»  . 

Stationary  random  processes  with  a  continuous  spectrum  are  not,  however,  the 
only  such  processes.  There  exist  stationary  (in  the  wide  sense)  processes,  the  cor¬ 
relation  functions  of  idxich  with  an  \mlimited  increase  of  v*  tend  toward  a  constant, 
or  are  periodic  functions  of  zr ,  This,  of  course,  violates  the  first  condition  of 
(5.45)  and  the  power  spectrum  ceases  to  be  a  continuous  function  of  the  frequency. 

Let  us,  for  instance,  examine  the  random  process 


^(0 = cos  wj/ + sin  »,/, 


(5.68) 


where  and  are  random  variables  not  dependent  on  t,  and  is  a  constant. 

It  can  be  said  that  the  random  function  4  (t)  represents  •’harmonic  vibrations** 

with  a  random  amplitude  of  A  and  a  phase  of  tf  =  arc  tg  ,  the 

t  z  •  $  J 
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distributions  of  wliich  do  not  depend  on  time. 

Random  process  (5.68)  will  not  always  be  stationary.  For  its  stationarity  (in 
the  wide  sense)  it  is  necessary  that  the  following  conditions  be  fulfilled.  First, 
the  mean  value  of  the  process  nay  not  depend  on  time,  inasmuch  as 

Wi  {S(01  =wi  {?ij  cosa>,/+m,  {?,)  sino),/. 


this  condition  nay  only  be  fulfilled  when 


(5.6^) 


i.e.,  when  the  mean  values  of  the  random  variables  %  and  ^  are  equal  to  zero.  In  the 

11 

second  place,  the  correlation  function  of  the  process  nay  depend  only  on  the  one 
parameter  T  ,  inasmuch  as 

(^‘') — '"i  {5  (0  5  + ■')} = cos  u>,/  cos  ci>,  (/  -j-  t)  4- 
4-  ^  sin  »,/  sin  m,  (/ + 1)  4- 1,5,  cos  o>,/  sin  ®,  (/  4- 1)  4- 
4" sin  cos  «i  (/  4' t)j  =  m|  }  cos  ® cos  ® j  (/4"*)4- 
4-  mi  1  sin  sin  •,  (/  4.  t)  4-  m,  sin  ®  i  (2<  4*  t), 

'  •  t 

B(t,z^)  will  not  depend  on  t,  if 


m,  {5,-^}=0. 


(5.70) 


(5.71) 


The  latter  is  fulfilled  if  and  are  independent. 

When  conditions  (5.69)  -  (5.71)  are  fulfilled,  a  harmonic  vibration  with  random 
amplitude  and  phase  will  be  stationary,  and  its  correlation  function  will  be  equal  to 

Bix)=jCosi»iX.  (5,72) 

In  this  case  the  amount  of  dispersion  of  the  random  ainplitude  will  be 

{5|4-t*}  =  m,  {5j}  4./n,{5|}  =  ,*. 


As  can  be  seen  from  (5.72),  the  correlation  function  of  vibration  with  a  random 
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amplitude  and  phase  is  proportional  to  the  dispersion  of  the  amplitude,  but  does  not 
depend  on  any  statistical  characteristic  of  the  phase. 

Although  correlation  function  (5*72)  does  not  satisfy  inequality  (5. *♦■5).  the 
concept  of  the  power  spectrum  can  nevertheless  be  expanded  also  to  the  case  at  hand, 
with  the  employment  of  the  delta-function  (cf.  J^pendix  IV).  Then  the  Fourier 
transformation  of  correlation  function  (5»72),  i.e.,  the  power  spectrum  of  vibration 
with  a  random  amplitude  and  phase,  may  be  represented  in  the  form  of 

+ 5(u)— 0),)].  (5.73) 

This  spectrum  consists  of  two  discrete  lines  of  unlimited  intensity  at  the  frequen¬ 
cies  of  + 

A  further  generalization  is  a  random  process  which  is  the  superposition  of  n 
elementaiy  stationary  random  processes  of  the  (5.68)  type: 

A 

MO = J]  %  cos  /  +  ’J*  sin  0)*  t).  (5.74) 

k»t 

The  conditions  of  stationarity  (in  the  broad  sense)  of  this  process  are  analo¬ 
gous  to  (5.69)  -  (5.71).  The  correlation  function  of  process  (5.7^)  takes  the  fol¬ 
lowing  form  when  the  conditions  of  stationarity  are  fulfilled: 


where 


cos  iO  T, 


(5.75) 


% 


5* 


I 


The  power  spectrum  of  stationary  process  (5.7^).  i.e.,  the  Fourier  transforma¬ 
tion  of  correlation  function  (5.75).  is  the  sum  of  the  delta-functions  at  the  discretl 
frequencies  of  +  w 


(5.76) 


A  power  spectrum  of. the  type  of  (5.76)  is  called  discrete,  and  the  stationary 


random  process  to  which  it  corresponds  is  called  a  process  with  a  discrete  spectrum, p 
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It  Is  evident  from  e  comparison  of  (5»75)  and  (5.22),  that  the  sum  of  S(t) 


harmonic  vibrations  with  frequencies  of  and  amplitudes  of  Cj^  has  the  same  auto¬ 
correlation  fxmction,  and  consequently  the  same  power  spectrum,  as  a  stationary 

random  process  representing  the  sum  of  the  same  frequencies  with  random  amplitudes 

2 

and  phases,  the  dispersions  of  the  random  amplitudes  coinciding  with  Cj^  .  This  com¬ 
parison  emphasizes  once  again  the  fact  that  the  correlation  function  provides  only 
an  average  power  concept  of  a  process,  without  taking  momentary  values  into  account, 
as  a  result  of  which  it  could  turn  out  that  two  processes  which  are  different  in 
principle,  have  the  same  correlation  functions  and  power  spectra. 

A  stationary  random  process  with  a  discrete  spectrum  of  the  (5.7^)  type 

sw-f;  (s.  cos  ««!•(). 

where  ^  and  71  are  random  variables  subject  to  the  conditions  of 
K  IK 

[o  (k^n), 

is  sometimes  regarded  as  an  approximation  of  a  stationary  random  process  with  a 
ccntf.n’ious  spectrum.  This  anprcxl nation  signifies  the  following.  The  realization 
of  a  random  process  with  a  continuous  spectrum  is  observed  during  the  time  NT.,  where 
N  is  very  large.  Then  the  total  period  of  observation  is  broken  down  into  N  equal 
parts,  each  with  &  duration  of  T.  For  each  of  the  indicated  time  segments  T,  the 
realization  of  the  random  process  may  be  resolved  into  a  Fourier  series 


r=l,  2,...  W. 

The  coefficients  Siid  change  with  a  transfer  from  one  segnent  to  another.  The 
aggregates  of  the  values  of  these  coefficients  are  the  random  variables  and  7^^  in 
(5»7^).  It  is  clear,  that  the  larger  is  T,  the  better  is  the  indicated  approximation. 
This  approximation  may  also  be  represented  in  another  form 
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VB.th  such  a  representation,  ^ (t)  is  regarded  as  a  sum  of  harmonic  vibrations  with  J 
random  amplitudes  and  phases.  | 

10.  Narrow-band  and  Vfl.de-'oand  Processes.  % 

~  ' 

A  case  of  practical  Importance  is  when  the  power  spectrum  ?((0)  of  a  random 
process  is  primarily  concentrated  in  a  relatively  narrow  frequency  band  about  a  | 

i' 
t  A 

certain  fixed  high  frequency  ,  where  the  spectral  density  is  at  its  maximum  (fig.  I 
42).  Then  the  correlation  function  B(2r)  of  such  a  random  process  may  be  represented 


in  the  form  of 


09 

B(t)=  f  (u)Cos  u>tdu»= 

,  ^ 

*=27  J  — “)cos((»o  — a>)xdu>  = 
— 0» 

= J  ^  (“o  “•  “>)  cos  ®t  d®  j  cos  Wo'  + 
— 0* 

4*[-^  J  ^(®o  ■"  •)  sin  »T  du  j  sin  (OqT, 


(5.77) 


Since,  according  to  the  assumption,  the  spectrum  band  is  relatively  narrow  in  com¬ 
parison  to  ,  the  upper  limits  of  integration  in  (5.77)  may  without  appreciable 
error  be  exteiided  to  infinity. 


Designating 


«0 

'  "  27  J  (*)  cos  Wt  dw. 


(5.78) 


(5.79) 


we  obtain 


m 

A(t)=^  Jf*(w)sinwTdw, 


(5.80)  i 


fi  (t)  =:  a  (t)  cos  Wo’t + b  (t)  sin  Wjt. 


(5.81) 


In  virtue  of  the  fact  th<.t  power  spectrum  F(»}  is  concentrated  in  a  narrow  || 

j| 

frequency  band  about  6)  =  W  ,  the  spectrum  F(m  -«)«?♦(«)  lies  in  a  low-frequent  I 
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r<mg..  Then,  ae  can  be  seen  fmn  (5.79)  and  (5.80),  a(r)  and  b(r)  sill  be  slowly 
functions  of  r  .  Here.  If  the  bandwidth  of  the  low-frequencjr  spectrum  is 
of  the  order  of  d  ,  the  correlation  function  will  have  a  "width"  on  the  order  of 
t/d  .  It  Hu)  nay  bo  considered  symetrlcal  with  respect  to  the  central  frequency 

u  ,  then  b(r)  =  0,  and  from  (5.81)  ve  obtain 

0 

fl\t)=a('c)cos«i)jT=^-^  Jr*(o))cosu»tdu»jcosM)jt.  (5,82) 

0 

Consequently,  the  correlation  function  of  a  narrow-band  process,  whose  spectrum 
is  symmetrical  about  high  frequency  is  equal  to  the  correlation  funcUon  a(r) 
multiplied  by  cos  which  corresponds  to  spectrum  F*(ai)  obtained  from  the  origi¬ 
nal  one  through  displacement  by  the  amount  of  into  the  low-frequency  range. 


fig,  42*  Harrow-band  and  wide-band  power  spectra. 

In  contrast  to  the  narrow-band  spectrum  examined  above,  let  us  now  examine  the 
power  spectrum  of  a  random  process  with  a  very  wide  band  (Fig.  42).  let  the  spectrum 
density  7(0)  of  the  mean  power  of  the  random  process  maintain  a  constant  value  up  to 
very  high  frequencies.  Ihe  correlation  function  B(r)  of  such  a  process  will  differ 
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f««  wro  only  In  a  aery  enall  Int.raal  of  the  «1«>  of  U.  arp-ent  about  the  ori¬ 
gin  of  the  eoordlnatea.  i.e..  for  anail  inatanooa  of  r  .  the  power  apeetrun 


f(u))=fj= const, 


(5.83) 


unifom  at  all  fre<iu«.cies.  la  a  uaeful  n.thenatloal  Idealiaation  of  the  apeotra  of 


the  Indicated  type. 


A  random  process  with  a  unifom  spectrum  at  all  frequencies  is  called  -white 


noise"*.  The  correlation  fuaiction  of  white  noise  is  equal  to  (. 

—0* 

i.e.,  is  a  delta-fhnction  at  the  origin  of  its  coordinates  (cf.  appendix  17). 


(5.84) 


The  correlation  coefficient  for  white  noise  is 


\0  t^O. 


(5.85) 


Thus  the  white  noise  $ (t)  is  characterized  by  the  fact  that  the  values  of  ^  (t) 
at  any  two  time  instants  (no  matter  how  close  together)  are  uncorrelated.  For  this 
reason  white  noise  is  sometimes  also  called  an  absolutely  random  process. 

It  should  be  emphasized,  that  the  white  noise  concept  defines  only  the  spectrum 
picture  of  a  random  process  and  leaves  cou^plotely  open  the  question  of  the  laws  of 
distribution.  Since  the  power  spectrum  does  not  uniquely  define  the  laws  of  dis¬ 
tribution.  white  noise  may  be  called  random  processes  which  have  a  unifom  power 

spectrum  and  various  laws  of  distribution. 

Ihus,  white  noises  are  an  exanple  of  those  random  processes  whose  correlation 

functions  and  power  spectra  are  the  same,  and  whose  distribution  laws  may  differ  from 
one  aiKjther. 

If  a  sequence  of  base  pulses  is  employed  as  an  approximation  to  the  stationary 

random  process,  the  case  where  the  amplitudes  of  these  pulses  are  mutually  inde- 

pendent  will  correspond  to  white  noise.  For  this  reason  various  types  of  white  noise 

•  m  analogy  to  white  light,  which  has  a  continuous  and  approximately  homoge.nous 
spectrum  within  the  limits  of  its  visible  portion. 
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must  be  fully  characterized  by  only  a  one-dimensional  distribution  function.  It  is, 
for  instance,  possible  to  speak  of  white  noise  with  a  normal  distribution  law  or  of 
white  noise  with  a  Rayleigh  distribution  law. 

White  noise  is  an  idealization  never  realized  under  actual  conditions  since,  in 
the  first  place,  sufficiently  close  values  of  a  random  function  are  practically  al¬ 
ways  dependent  and,  secondly,  real  processes  have  finite  power,  while  for  white  noise 
the  full  power  of  the  process  is  infinite.  However,  as  a  result  of  the  limited 
nature  of  the  pass  bands  of  radio  equipment,  such  an  idealization,  idiile  consider¬ 
ably  simplifying  the  mathematical  analysis  of  the  processes,  introduces  no  errors  of 
any  significance^. 

11.  Ihe  Normal  Random  Process. 

Some  general  statistical  characteristics  of  stationary  random  processes  have 
been  cited  above,  and  their  properties  have  been  indicated.  The  special  type  of  these 
processes  most  frequently  encountered  in  radio-engineering  applications  is  the  so- 
called  normal  random  process. 

The  x*andom  process  ^(t)  is  called  normal  (gaussian)  if  the  distribution 

functions  cf  any  n-th  order  for  the  aggregate  of  random  variables  ^  (k=1 , 

K 

2,.,.n)  are  nonaal,  l.e,,  are  determined  by  the  foraulas  (cf,  (.Z.liO)) 

(Xj, );=  777====--  --  X 


where 


*”20  S  2  **  »j  •* 


«*— «i 


(5.86) 


R\2 

R32 

(5.87) 

(5.88) 

(5.89) 


•  Strictly  speaking,  the  concept  of  the  value  of  the  random  function  $(t),  at  the 
point  t,  itself  ceases  to  make  sense  for  idiite  noise  and  it  is  possible  to  speak 
only  of  the  results  of  tbie  passage  0^“  white  noise  through  linear  systems,  as  will 
be  done  ir  the  analysis  of  tha  foi:  ..ig  chapter  (cf.  also  [19]  ). 
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If 

'i 


■  ){ 


The  quantity  Dj^j^  is  an  algebraic  co-factor  in  the  deteminant  D  of  the  element 
idiich  is  the  correlation  coefficient  of  the  random  variables  ^(t;)and  ^  (t^), 

'  «  _«iH€(^i)— *1}  «(/,./*)  _  _ 

(5.90) 

If  all  the  mean  values  aj^  and  the  dispersions  are  constant  (i.e.,  do  not  de¬ 
pend  on  the  index  k), 

a*=fl.  o,=a. 


and  the  correlation  function  B(t^,  t^)  depends  not  on  the  two  variables  tj^  and  tj^ 
but  only  on  their  difference  X^y  ~  •  ^k*  distribution  function  (5»86)  does 

not  change  with  any  displacement  of  the  entire  group  of  points  *  constant 

amount  along  the  tine  axis,  i.e. ,  with  the  fulfillment  of  the  indicated  conditions, 
a  normal  random  process  is  stationary.  Here  a  distribution  function  of  the  n-th 
ordar  will  depend  only  on  the  n-1  parameters  ^  “  I  I 


(5.92) 


¥  (2«9i}*D 


where  a  and  «r  are  constant  numbers,  while  D  and  are  numerical  functions  of  the 


parameters  ...  These  functions  are  deterrined  by  the  values  of  the  cor- 


relation  coefficient  R  (r)  = — yi;—  with  toe  indicated  n-1  values  of  Z  , 


Since  for  a  stationary  process  the  cor  illation  coefficient  is  an  even  function, 
therefore  in  deteminant  (5.89),  for  a  sta  .ionaxy  nomal  process,  Sj^j^  =  RCo*)  »  1  and 
®ki  *  ®ik  *  i.e..  the  elements  of  the  principal  diagonal  of  D  are  equal  to 

unity,  and  those  of  its  elements  which  are  symmetrical  with  respect  to  this  diagonal 
are  equal  to  each  other. 

Thus,  in  order  to  determine  the  distribution  function  of  a  normal  random  process, 
of  any  order,  it  is  sufficient  to  know  only  its  correlation  function.  If  this  cor¬ 
relation  function  depends  00  one  argument  r  (stationarity  in  the  wide  sense),  the 
nomal  process  is  stationai^'^  in  the  precise  sense.  Komal  stationary  processes  may 
differ  from  each  other  in  the  fom  of  their  correlation  function  or  of  their  power 


Xti 
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spectrun,  given  in  each  concrete  problem  on  the  basis  of  supplementary  conditions. 

If  two  random  normal  processes  arc  incoherent  i.e,,  if  their  mutual  correlation 
function  is  equal  to  zero,  then  they  are  independent  (which,  as  has  been  noted  above, 
is  not  valid  in  the  general  case). 

The  first  three  distribution  functions  of  a  stationary  random  process  with  a 
zero  mean  value  have,  in  accordance  with  (2.14),  (2.15),  »nd  (2.62)  the  following 
form: 


»*(*!.  *1.  ‘»)-* 


t 

J»f  +jr|-*a  (tJjTi^r, 


w,(x)=-^e 

'  K2Ki* 


•,(X,.  X,.  X„t,,t,): 


(2«*)*'»  |/  1  _  _  ft*  ~  +  2ft, ft, ft, 


(5.93) 

(5.94) 

(5.95) 


where 


0--^|)*i4-(l-/??)x|-|-(>-.ft»)xg 

2e*  (1  _  ft*  -  ft*  _  ft*  +  2ftift,ft,) 


_  2(^1-  ^ift,)  xix,4-2  (ft,-  ft.ftO  xtx^+2  (/?,-fttft,)  x.x. 
2«i  (I  ~  ft*- ft*- ft*  + 2ft,ft,ft,) 


/?l  *=  Rii  ss»  i?2l  ~  ^  ('l)» 

/?J  =  Rfy  =  /?„  =  R  (t.,), 
^9—Rii—Ri3=R  (tj  + 1,). 


A  normal  rindom  process  may  be  a  idiite  noise,  if  the  spectrum  of  the  process  is 
luiiform  at  all  frequencies.  Then  the  correlation  coefficient  R  (t)  is  determ? ned  ac¬ 
cording  to  formula  (5.85)  and  consequently,  v'hile  from  (5.89) 
we  find  D  ^  1  and 


D  — ^ ^ 
"  \0. 


The  distribution  function  of  the  n-th  order .  in' this  case,  in  accordance  with*- 
(5*92),  has  the  form  of 


I_e  *-* 


(2«*) 
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l*e,,  is  the  product  of  n  one-dimensional  distribution  functions,  which  corresponds 
to  the  independence  of  momentary  values  of  white  noise  at  any  two  moments  in  time. 
The  normal  stationary  random  process  with  a  uniform  spectrum  embraces  a  broad 


class  of  phenomena  observable  in  radio  apparatus.  *1 

Baploylng  the  central  limit  theorem,  it  is  possible  to  show  that  the  so-called 
internal  (fluctuation)  noises,  caused  the  shot  effect  and  by  the  thermal  motion  | 
of  electrons,  have  a  noraal  law  of  distribution,  which  is  also  confirmed  experiment¬ 
ally.  The  spectrum  of  these  noises  is,  according  to  various  data  [?!  ,  practically 
imiform  to  frequencies  on  the  order  to  10^^  cycles,  which  goes  far  beyond  the  limits  C 

of  that  frequency  range  irtiich  is  of  interest  in  contemporary  radio-engineering  ? 

problems.  > 

.  >• 

The  fact  that  a  normal  distribution  may  be  obtained  through  the  addition  of  a  ! 
large  number  of  independent  vibrations,  is  employed  in  experimental  practice  for  the 
modelling  of  normally  distributed  noises  through  summation  of  the  sinusoidal  j 

vibrations  of  independent  generators  with  appropriately  selected  amplitudes  and  ^ 

i 

Let  us  note  that  certain  types  of  internal  noise  do  not  have  a  normal  distri-  ;p 

button.  Among  these  are,  for  instance,  contact  noise,  which  takes  place  when  cuiTeat 

I  I 

travels  along  a  boundary  between  two  conductors,  as  well  as  noise  In  sc'^t -conductors  ,  j 

'  I 

(transistors)  [?]  .  •  I 

A  stationary  normal  random  process  with  a  uniform  spectrum  in  a  frequency  band  || 

■  I 

of  the  order  of  4  kc  may  be  employed  as  an  approximate  model  of  a  modulating  voltage 
corresponding  to  the  transmission  of  human  speech.  The  validity  of  such  modeling  I 
has  been  confirmed  by  several  experiments  [  8] ,  | 

It  is  all  the  more  permissible  to  consider  as  normal  the  .probability  distribution  I 
of  instantaneous  values  of  multichannel  communication  (with  frequency  division  of  j 
channels)  which  is  the  sum  of  a  large  number  of  independent  speech  vibrations.  {  | 

It  is  often  necessary  to  study  processes  which  are  a  sum  of  the  fluctuation  | 

noises  (normal  stationary  random  process)  ^(t)  and  the  determined  signal  S(t),  1 
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Bearing  in  mind  (5.8),  we  find  that  the  distribution  functions  of  the  sum  ^  (t),  S(t) 
are  also  normal  and  are  obtained  from  (5*92),  if  in  the  exponent  of  each  of  the  bi¬ 
nomials,  Xj^  -  a  is  replaced  by  Xj^  -  a  -  S(t).  Therefore  the  sum  of  signal  and  noise 
at  hand  is  a  normal  stationary  process  which,  however,  is  not  stationary  in  the 


strict  sense. 


12.  Derivative  of  the  Normal  Random  Process. 


If  the  correlation  function,  and  its  second  derivative,  of  a  stationary  random 
process  are  continuous  iTheh  •  0,  then  in  accordance  vriLth  Section  8  this 

process  will  be  continuous'  luid  differentiable. 

Let  us  make  use  of  formula  (5.^1)  in  order  to  find  the  first  distribution 
function  of  the  derivative  of  a  normal  random  process.  The  starting  point  is  the 
expression  for  the  two-dimensional  distribution  function  of  process  ^  (t) 


(^1.  J.  t)  =  -  ■  . _ e 

2*ci  y  I— /{I  . 


(5.96) 


where  R  =  R(2')  is  the  correlation  coefficient,  .  ’’  • 

_ The  distribution  function  t ).  of  the  difference  ^(t  ftr)  -.-^(t)  is  equal  to 

■  « 

ir,(y,t)=  — li  fc  •’“+'<>  du--= 

'4  J  2it»*  Kl  — R*  J 

,  -  >* _ ?  (“-*+4)' 

s^ctyjzijr  *  Jc  du. 


and,  noting  that  the  integral  is  equal  to  /Sir  ff  ,  we  obtain 


I  J'* 

y|(y.T)=—  ■* 


(5.97) 


It  is  now  not  difficult  to  find  the  distribution  function  of 


which  is  equal  to 
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(yto)  =s  ->.= -^==  e~ 


(5.98) 


The  desired  distribution  function  of  the  derivative  (y)  is  the  limit  of  (5*98) 


when  V—*  0,  Since  whentT— »0  rCi*) — >1,  an  indeterminaqr  results  both  in  the  numer¬ 


ator  and  the  denominator.  For  resolution  of  this  indeterminaqy,  we  break  down 


function  R('?)  into  a  Maclaurin  series 


(0)  -ft/?'  (0)  -f  ~  R"  (0)  -f  0 


But  R(0)  =  1,  and  R'(0)  =  0  and  R'*(0)<0,  since,  when  f  =  0,  the  function  R(tr)  is  at 


its  maximum.  Therefore 


I  -/?  (t)  =  -  ^  /?"(0)  -f  0  (t3). 


(5.99) 


Substituting  (5.99)  into  (5.93)  and  taking  the  limit,  we  obtain 


W’'"W=llmi4=. 


t«0  K2*al  K_/{"(0)+O(t) 


■  2.*  1-R»  (0)+Ot^)I 


wherefrom 


where  it  is  specified  that 


y* 

-_i|%  .  -  I  29^w2 

-4=6  j 

2m*  wj 


(5.100) 


u,»=_/?"(0),  r(0)<o. 


(5.101) 


Thus  the  one-dimensional  distribution  function  of  the  derivative  of  a  statlon- 


%  X.  X 

aiy  normal  process,  is  normal,  with  a  zero  mean  value  and  a  dispersion  of  o'  ^*<r 


|a’’(o)  I . 


It  is  possible  to  show  that  the  two-dimensional  distribution  function  of  the 


derivative  of  a  stationary  normal  process  is  also  normal,  and  that  its  correlation 


function  is  {%)  -  -B"(r), 


Thus  the  random  process  obtained  by  differentiating  a  stationary,  normal  process 


with  a  correlation  function  of  B(t)  is  stationary  and  normal,  with  a  correlation 


function  of  -B'’(‘t). 
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Let  us  also  define  the  Joint  distribution  function  of  a  stationary  normal  random 
process  and  its  derivative  in  coincident  instants  of  time.  Etaploying  (5*67* )  and 
(5«100),  we  find  that 


I  •  1 

/2r.et«2 


(5.102) 


The  Joint  distribution  of  ^(t),  ^^(t),  ^(t  +r),  is  also  normal,  and 

the  determinant  corresponding  to  this  distribution  (5.89)  has  the  form  ^cf,  (5.64) 
and  (5.66)]  of 


1  0  RiS)  R’{x) 

0  1  --R'(t) 

/?(^) -«'(')  1  0 

/?'(t) -/?'(')  0  1 


(5.103) 


o  r' 


13.  Average  Number  of  Intersections  of  Given  Level  in  Unit  of  Time, 


In  solving  a  number  of  practical  problems  it  is  necessary  to  know  the  distribu-  ? 

- 

tion  function  for  the  overshoot  duration  of  a  random  process  $  (t),  where  by  over- 

A 

i 

shoot  duration  is  meant  the  segment  of  time,  during  which  ?  (t)  exceeds  an  assigned  1 

t 

level  X  =  Xq.  Sometimes  instead  of  the  distribution  of  overshoot  duration  one  speaks  1 

t 

of  the  null  distribution  of  a  xandom  function,  i.e.,  the  intersection  of  thia 

functi'-n  'dth  -n  j.s£lgned  h.rii..''nta?  straight  line.  the  general  case  there  can  oe  | 

A  i 

examined  the  problem  of  determining  the  distribution  of  the  points  of  intersection  1‘ 
of  function  9  (t)  with  an  assigned  function  f(t),  A  full-scale  solution  of  the  f j 

problem  under  consideration  is  very  complex,  but  a  number  of  useful  approximations 
in  special  cases  has  been  obtained  by  several  authors  [10  -  I3J  *  if 

‘S 

Let  us  restrict  ourselves  to  the  simplest  statistical  characteristic  of  the  over! 
shoots  of  a  random  process  —  the  average  number  of  the  intersection  by  this  process  j 
of  the  assigned  level  x  =  x^  in  a  unit  of  time.  j 

i 

The  probability  of  intersecting  level  x  =  x_  from  below  (i.e.,  with  a  positive  1 

i 

derivative)  with  a  sufficiently  small  At>--2I^ —  coincides  with  the  probability 
of  the  inequalities  :r 
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*o~Ax<UO<*o.^>0. 


Let  w,(x,  y,  t)  be  a  two-dimensional  distribution  function  ct  4  it)  and  at 

the  coincident  time  instant  t. 

Then 


P  Ax<l(0  <  Xo.  >0  ]  = 


=  f  J  »a(*.  y.  t)dxdy. 

0  Xt—ix 


With  a  sufficiently  small  A  t,  the  inside  integral  may  be  replaced  by  the  ex¬ 
pression  w^Cx^,  y.  t)A  X  =  yWgCxQ,  y,  t)At  and 

00 

P  Jxj  —  Ax  <  E  (0  <  Xo.  ^  >  0 1  =  A/ 1  ytt»2  (Xq,  y.  t)  dy. 


(5.104) 


The  average  number  of  the  intersections  of  level  x  ~  during  the  time  T  is 
equal  to 

l^i(Xo.r)=j  P{xo--Ax<E(/)<Xo,^>>oJ= 

n-  (5.105) 

y.  Odydf, 

\ 

and  the  average  number  of  the  intersections  of  this  level  in  a  unit  of  time  (with  a 
positivts  derivative)  is  equal  co 


/i|(Xo,  T) 


_Af,(Xn.n  1 


/4-ro» 


J  jy«'2(xo,y.O<fyd(- 


(5.106) 


t  0 


For  a  stationary  random  process  ^ (t),  a  joint  distribution  of  ^ (t)  and  its 
derivative  at  a  coincident  time  instant,  t,  does  not  depend  upon  t,  and  the  mean 


number  of  intersections  of  the  level  x  =  per  unit  of  time  equals 


«i(j^)=J>w2(*o.  y)<^y- 


(5.107) 


ilttth.  of  (5V67* ),  it  is  possible  to  represent  ^ 
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of 


n,(Xo)= 


W|(jf#)  j* 


y*»r(y)rfy=wi+V(x,), 


(5.t08) 


iihere  is  »n  average  of  the  positive  values  of  derivative  4**(t). 


Thus  the  curve  for  the  dependence  of  the  average  nunber  of  overshoots  on  the 


level  of  Xq  coincides  with  the  one-dimensional  distribution  curve  of  ^  (t)  with  a 


precision  given  by  a  constant  inultiplier*  So,  for  instance,  the  average  number  of 


overshoots  of  a  normal  random  process  is  on  the  basis  of  (5*100)  equal  to 


(5.109) 


In  a  completely  analogous  manner  is  obtained  the  formula  for  the  average  niunber 


of  intersections,  in  a  unit  of  time,  of  level  x  =  x^  from  above  (i.e.,  with  a  nega¬ 
tive  derivative) 


0  '  0  ’ 

«a  (Xo)  -  -  J  (Xg,  y)  dy  =  J I  y  1  y)  dy, 


(5.110) 


"2(Xo)=  m\  ’j®,(Xo). 


(5.110*) 


(—)  *• 
where  m^  is  an  average  of  the  negative  values  of  derivative  v  (t). 

The  overall  average  number  of  intersections  of  $  (t)  with  the  straight  line 


X  3  X  (the  average  number  of  nulls  of  a  random  function)  is  equal  to  the  sum 


rt  (Xg)  =  n,  (Xg)  4-  rt,  (Xg)  =  J I  y  I  tCj  (Xg,  y)  dy. 


(5.111) 
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Chapter  VI 

TRANSFDRMATIONS  OF  RANDOM  PROCESSES 


1,  Definition  and  Terminology 

A  considerable  number  of  radio-engineering  problems  consist  in  analyzing  the 
results  of  the  action  of  certain  processes  on  devices  of  a  greater  or  lesser  degree 
of  complexity.  Setting  aside  a  detailed  characterization  both  of  active  processes 
and  of  radio-engineering  ^sterns,  we  shall  only  pause  briefly  on  the  basic  defi¬ 
nitions  and  terminology  to  the  extent  required  for  the  forthcoming  exposition. 

An  active  process  is  the  sum  of  a  desired  sigiial  and  of  the  noise  distorting  it; 
each  of  the  terms  of  this  sum  may  be  either  a  given  function  of  time,  or  a  purely 
random  process.  In  modern  general  commimi cations  theory,  for  instance,  transmitted 
signals  are  regarded  not  as  given  functions  of  time,  but  as  an  aggregate  of  possible 
functions  of  time  which  have  definite  statistical  properties. 

The  signal-distorting  noise  may  be  an  internal  noise  of  radio  transmitting  arid 
receiving  devices,  or  external  noise  (man-made,  or  atmospheric  and  industrial). 

It  is  customary  to  divide  elements  of  radio  devices  into  two  fundamental  groups; 
nonlinear  inertialess  and  linear  (inertial  or  dynamic).  The  first  group  includes 
such  elements  as  modulators,  detectors,  limiters,  and  mixers,  the  second  —  ampli¬ 
fiers  and  filters*. 

The  characteristic  of  a  nonlinear,  inertialess  element  is  given  in  the  form  of 
the  single-valued,  non-linear  transformation 

y(t)  =  f  [x(t)]  , 

in  which  X  refers  to  the  input,  and  y  to  the  output,  the  value  of  y(t)  at  a  given 
moment  of  time  t  being  determined  by  the  value  of  x(t)  only  at  the  same  moment  of 
time  t. 


*  It  is  known  that  such  processes  as  modulation,  detection  and  transformation  may 
take  place  by  means  of  linear  systems  with  variable  parameters.  Within  the  frame¬ 
work  of  the  present  book,  only  linear  systems  with  constant  parameters  will  be  ex¬ 
amined. 
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A  linear,  inertial  network  is  characterized  by  the  transfer  function  k(i«i)),  the 
Kodulus  and  argument  of  which  are  respectively  the  frequency  c(e))  and  phase  («) 
characteristics  of  this  network. 

In  place  of  the  frequency  and  phase  characteristics,  often  the  so-called 
transient  pulse  function  h(t)  of  a  linear  network  is  employed,  linked  with  the 
transfer  function  ty  the  Fourier  transformation 


J  A(He'“'du,. 


(6.1) 


Elaploying  the  function  h(t),  it  is  possible  to  express  the  process  y(t)  at  the 
output  of  a  linear  network  in  terms  of  the  process  at  the  input  x(t),  by  means  of  a 


relationship  known  as  Duhamel^s  integral 


y(0=Jx(/-t)A(t)dT, 


(6.2) 


m  1  f 
^  r 


from  which  it  can  be  seen  that  the  value  of  y(t)  at  a  given  moment  of  time  t  depends 
on  the  values  of  x  preceding  the  moment  t.  If,  besides  that,  x(t)  S  0  when  t  <  0, 


y(0=|x(f~t)A{t)d. 


(6.2*) 


The  lover  limit  of  integration  in  (6.2)  is  taken  as  equal  to  zero,  and  not  to 
—  ,  since  from  the  condition  of  the  physical  feasibility  of  the  linear  network, 

h(t)  %  0  when  t  <  0. 

Strictly  speaking,  a  nonlinear  device  (for  instance,  a  detector)  contains  not 
only  an  element  defined  by  a  nonlinear  functional  relationship,  but  also  a  linear 
inertial  element  (the  electrical  circuit  as  detector  load).  On  the  other  hand,  the 
volt-ampere  characteristics  of  linear  networks  presei^e  their  linearity  only  within 
specific  boundaries.  Thus  any  element  of  radio  apparatu;  should  as  a  matter  of 
principle  be  regarded  as  nonlinear  and  inertial.  However,  the  existing  mathematical 
apparatus  is  not  in  a  condition  to  overcome  those  difficulties  which  arise  in  the 
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solution  of  problems  with  such  general  assumptions.  Therefore  the  indicated  division 
of  radio  apparatus  into  linear  and  nonlinear  remains  expedient  for  the  time  being. 

The  error  from  such  an  idealization  may  be  assessed  for  each  concr<>te  problem*  Thus, 
for  instance,  if  the  plate  load  of  a  tube  consists  predominantly  of  resistance,  the 
inertia  of  its  characteristic  may  be  neglected.  A  detector  vith  a  load  in  the  fon# 
of  a  selective  circuit  may  with  a  certain  degiree  of  approximation  be  regairded  as  a 
series  connection  of  a  rionlinear  non-inertial  network  (rectifier)  and  a  linear  in¬ 
ertial  network  (load). 

The  total  useful  effect  produced  by  a  nonlinear  device  is  obtained  as  a  result 
of  two  transformations,  of  irtxich  one  is  nonlinear  and  non-inertial,  while  the  other 
is  linear  and  inertial. 

In  the  following  sections  are  examined  the  trans  ;*ormations  of  random  processes 
in  their  passage  through  linear  and  nonlinear  networks. 

In  this  connection  the  problems  examined  can  be  of  two  types*. 

1 )  Determination  of  the  correlation  function  and  power  spectrum  of  a  process 
at  the  output  of  a  network  (ndnimun  problem). 

2)  Determination  of  the  multidimensional  distribution  functions  of  a  process 
at  the  output  of  a  network  (maximum  problem). 

It  is  clear  that  from  a  solution  of  the  second  of  the  indicated  problems  a 
solution  of  the  first  can  be  obtained. 

Actual  radio  apparatus  consists  of  a  whole  series  of  linear  and  nonlinear  ele¬ 
ments.  For  this  reason  a  complete  solution  of  the  problem  will  require  several  con¬ 
secutive  transformations  of  the  random  process. 

A  typical  unit  of  radio  apparatus  is  a  link  consisting  of  three  elements:  two 
linear  networks,  between  which  is  located  a  nonlinear  system  (Fig.  43). 

c!(u}  c}(u) 

(Unear  j,Hon-  [linear  ] 

-•  netvo^  networkr 

tinput;  network  (output) 

'  Fig.  43.  Standard  radio-apparatus  link. 
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The  transformations  in  such  a  standard  link  are  characteristic  of  many  stages 
of  radio  transmission  and  reception  (frequency  transformations,  modulation,  detection, 
limiting,  etc.). 

In  this  chapter,  the  general  methods  of  solving  the  problems  formulated  above 
will  be  examined  separately  for  linear  and  nonlinear  networks.  The  application  of 
these  methods  is  illustrated  in  the  following  ch^tors.  The  question  of  the  trans¬ 
formation  of  distribution  hinctions  in  the  standard  link,  if  its  input  is  acted  upon 
by  a  normal  random  process,  is  examined  separately  in  Chapter  IX. 

2,  Power  Spectrum  and  Correlation  Function  of  Random  Process  at  the  Output 

of  a  Linear  Network 

In  accordance  with  (6.2),  a  linear  network  with  a  transient  pulse  function, 
h(7'),  transforms  the  random  process  (t),  fed  to  its  input,  .into  another  random 
process  ^^(^)  which  is  an  integral  of  the  first 

5,(/)=|5,((-t)/i(t)dT.  (6.3*) 

»  ***  t 

Let  Fj(<i»)  be  the  power  spectrum  of  the  stationary  random  process  ^^(t)  at  the 
input  of  a  linear  network  whose  frequency  characteristic  is  equal  to  C(w).  Then  at 
the  output  of  this  network  the  power  spectrum  of  process  ^^(t)  has  the  form  of* 


(6.4) 


Formula  (6.4)  is  the  formula  for  the  transformation  of  the  power  spectrum  of  a 
■^hationary  random  process  in  its  passage  through  a  linear  network  with  a  frequency 
chvacterisilc  of  C(6>). 

If  the  process  at  the  input  of  the  linear  network  consists  of  the  sum  of  a 
stationary  random  process  and  a  determined  process,  then,  bearing  in  mind  the  princi¬ 
ple  of  superposition  as  applicable  to  linear  networks,  it  is  possible  to  effect  a 


♦  A  rigorous  proof  of  formula  (6.4)  is  obtainable  from  an  examination  of  integral 

(6.3). 
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spectrum  transformation  accordins  to  (6.4)  separately  for  each  item,  and  then  to  sun 
up  the  results  obtained. 

As  vas  to  be  expected,  the  phase  characteristics  of  linear  networks  are  in  no 
way  reflected  in  the  transformation  formula  for  the  power  spectrum  of  a  process. 

From  (6.4)  it  follows  that  the  correlation  function  ®  process  at  the 

output  of  a  linear  network  is  equal  to 


0» 

^  Bj(‘c)=~J  F,(i*)C*(»)cos»tdo», 


(6.5) 


It  is  not  difficult  to  obtain  a  different  expression  for  B2(f),  employing  not 
the  input  spectrum  F^(<i>),  but  the  correlation  function  B^(f)  of  the  process  at  the 
input. 

2 

Let  us  designate  by  H(t)  the  Fourier  transformation  of  0  (W),  i.e,. 


Since 


»(0=s 


C»(«)=A(i»)A(-iu)), 


therefore,  with  (6,1)  in  mind,  we  find  on  the  basis  of  a  theorem  known  in  Fburier 


transfomntlon  theory  as  the  convolution  theorem,  that 


^(0=  jA(t)A(T  +  0‘^^. 


(6.6) 


The  Inverse  Fourier  transformation  of  F^(«)  is  B^(t);  therefore  on  the  basis  of 
the  convolution  theorem  we  obtain  from  (6*4)  the'expres^n  for  the  correlation ‘func¬ 
tion  of  a  random  process  at  the  outeut  of  a.  linear  netsrork 


m 


(6.7) 


Substituting  for  K(t)  its  expression  from  (6.6),  we  obtain  the  link  between  the  1  'M 

’  '  § 

input  and  output  correlation  fiinctlons  and  the  transient  pulse  function  of  the  net-  |*^ 
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(6.8) 


5a(')=  j  ^Bi{x-{-9  —  u)h{v)h{u)dvdu. 

From  (6.5)  ajid  (6.8)  there  follows  an  equality  useful  in  some  computations 


m 

«»  « 

=  J  ^Bi{o-'U)h\p)h(u)dvdu. 


(6.8* ) 


Bnploying  (6.7),  the  formula  for  the  spectrum  ?^(.u)  of  a  process  at  the  output 
of  a  network  nay  be  expressed  in  terms  of  the  correlation  function  B^(T)  of  the 
process  at  the  input 


F,(m)=:4  j  j 


Bt  («)  cos  mzdud'z. 


(6.9) 


Thus  the  problem  of  the  transformation  of  the  power  spectrum  of  a  random  process 
and  its  correlation  function  in  passage  through  linear  networks  is  fully  solved  by 
formulas  (6.4),  (6,5).  (6.8)  and  (6,9).  if  the  frequency  characteristic  (or  pulse 
function)  of  the  netvrork  and  the  spectrum  or  correlation  function  of  the  process  at 
the  input  are  given. 

In  formula  (6.3)  it  is  tacitly  assumed  that  the  cut-in  instant  of  4^^(t)  at  the 
input  of  a  linear  network  is  equal  to— .o©.  If  in  such  a  case  the  random  process 
$  j(t)  is  stationary,  random  process  ^^^(t)  at  the  output  of  the  linear  network  will 
also  be  stationary. 

In  cases  when  transients  in  linear  networks  are  being  studied,  it  is  necessary 
to  take  into  account  the  cut-in  instant,  i.e,,  to  assume  in  (6.3)  when  t  <  0,  that 
^jj^(t)s0.  The  upper  limit  in  integral  (6.3)  becomes  variable  and 

i~  ' 

<a(0=j5,(/— c)A(t)*.  (5,10) 

.Thus  the  transient  ^^(t)  in  a  linear  network  is  a  nonstationary  random  process,  even 
if  the  random  process  5^(t)  applied  to  its  input  is  stawj-onary.  The  average  value 
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of  the  process  at  the  output  of  the  linear  network  In  this  case  Is  equal  to 


*■  # 

and  if  $^(t)  is  a  ststionaiy  random  process,  then 

t  ^  ’ 

fla(0=atjA(t)rft=a,^(/)*.  (6.10* ) 

The  second  distribution  moment  of  the  process  in  this  case  also  is  dependent  on  time. 


and  in  accordance  with  (6.8*)  is  equal  to 


•  * 

”*tM=B,{0,i)==jjB^(v■^u)h{v)h{u)dvdu,  (6.10-) 

irtiere  B^(t)  is  the  correlation  function  of  the  stationary  random  process 

A  special  case  of  process  f ^(t)  when  h(r)  ~  1,  determined  on  the  basis  of 

* 

(6,10),  is  the  indefinite  Integral  of  the  random  process  ^.(t) 

t  * 

Since  to  the  function  h(r)=‘l there  corresponds  C(w)  -  therefore  the  power  spectrum 
for  the  integral  of  y^(t)  is  obtained  by  dividing  its  power  spectrum  hy  0)  ,  *.e., 

(6.11) 

Let  us  note  that,  employing  the  filtration  properties  of  the  derivatives  of  the 
del ta»f unction (cf« Appendix  IV),  it  is  possible  to  extend  Duham^'s  integral  (6*3)  to  dif¬ 
ferentiation  processes  of  random  functions.  Thus,  for  instance,  the  first,  derivative 
of  process  x(t)nay  be  regarded  as  a  result  of  the  passage  of  this  process  through  a 

linear  network  with  a  transient  pulse  function  equal  to  8'^^t).  It  then  follows  from 

(a) 

(6,6),  that  for  such  a  linear  network  H(t)  =  S  (t),  and  the  square  of  the  frequency 
characteristic  C^(w)  -  ^  • 


*>  The  function  A(t)  =  J  h(T')  d't  is  called  the  transient  conductance  of  a  linear 
network  or  the  reaction®of  a  linear  netwsrk  to  a  unit  step. 
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m- 


>'-Tc-^V,p.-.i-  ‘'V^-  ^'. 


En|)loying  the  general  formula  (6*U)  for  the  transformation  of  the  ]power  spectrum 
of  a  random  process  in  linear  nebrorks,  we  find  the  spectrum  of  the  derivative 

*£<^  F]^(ct))  that  corresponds  to  formula  (5*65)  which  was  obtained  on  the  basis 
of  other  considerations. 

3.  Passage  of  White  Noise  through  a  Linear  Network. 

Let  US  examine  the  case,  of  practical  Importance,  when  on  the  input  of  a  linear 
nettnrk  there  acts  a  random  process  with  a  power  spectrum  uniform  for  all  frequencies 
(white  noise)  (cf.  5*83)*« 

F|(6))  =  F^  =  const. 

Then  in  accordance  with  (6.4)  the  power  spectrum  F2  (w)  of  a  random  process  at  the 
output  of  this  network  is  equal  to 


^»(»)=FoC2(u.). 


(6.12) 


In  this  manner,  the  power  spectrum  of  white  noise  at  the  output  of  a  linear  net¬ 
work  coincides  in  shape  with  the  square  of  the  frequency  characteristic  of  the  net¬ 


work. 


From  (6.12)  we  obtain  directly  the  correlation  function  of  a  random  process  at 


the  output  of  a  linear  rstwor*' 


00 

^  (■')  =  ^  J  C*  (u))  cos 


(6.13) 


Bearing  in  mind  (6,5)  and  (6,6),  we  find 


•0 

=  ^h{u)/i{u^x)du. 


(6.13*) 


Thus  the  correlation  function  of  white  noise  at  the  output  of  a  linear  netwo’^k  coin¬ 
cides,  with  an  accuracy  of  to  the  last  constant  factor,  with  the  convolution  of  the 
transient  pulse  function  of.  the  network. 

*  An  exajiq)le  of  white  noise  is  the  fluctuation  noises  of  radio  receivers.  The  in¬ 
tensity  Fq  of  these  noises  per  unit  of  frequency  band  is  equal  to  n^  kT,  waere  k  is 
Boltzmann’s  constant,  T  is  the  absolute  temperature  (kT*4.10“21  watt/cps)  and  ny,  is 
the  noise  coefficient  of  the  receiver. 
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Por  a  narrow-band  linear  network  with  a  aynunetrical  frequen<^  characteristic 


relative  to  a  high  frequency  ,  the  correlation  function  at  the  output  may,  in 
accordance  with  (5«82),  be  represented  in  the  form  of 


m 

B  (t) =0  (t)  cos  j  Cj  (w)  cos  •trfu)  j  cos  ®gt, 


(6.14) 


whei-c 


It  can  be  seen  from  (6,14),  that  the  correlation  function  of  white  noise  which 
has  passed  through  a  narrow-band  linear  network  with  a  resonance  frequency  of  cJq  , 
is  equal  to  the  product  of  cos  CJgif  and  the  correlation  function  a('2r)  of  white  noise 
which  has  passed  through  a  network  with  a  frequency  characteristic  of  C^(a;),  obtained 
by  displacing  C(<J)  into  the  low-frequency  range  by  the  amount  6)^  , 

With  the  passage  of  white  noise  through  linear  networks  is  connected  the  so- 
called  power  determination  of  the  width  of  the  pass  band  of  their  frequency  character¬ 
istics.  According  to-  this  determination,  the  width  of  the  pass  band  is  computed  from 
the  full  power  of  the  white  noise  passing  through  a  linear  network,  the  actual 
frequency  characteristic  being  replaced  by  an  idealized  rectangular  characteristic, 
equivalent  to  the  power  of  the  noise  (Fig,  44),  The  height  of  the  rectangle  is  so 
selected,  that  the  constant  density  of  the  power  spectrum  within  the  limits  of  its 
width  equals  the  maximum  density  F(Ci)^)  of  the  actual  power  spectinm  at  the  output 
of  the  linear  network.  In  accordance  with  the  assumed  determination  the  width  of  the 
pass  band, A f,  of  the  frequency  characteristic  C(OJ)  is  computed  by  the  formula 


A/  =  . 


(6.15) 


or,  taking  into  accouuit  (6,12), 


Jc*  (»)</» 

2«C»K) 


(6.16) 
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m  I 


I  5 

fe  ’  i 


¥>  '  » 

I  i  j 

i  i  ! 


From  (6.15)  and  (5*^6)  there  follows  also 

,t_l  BIO) 
fiPK) 


(6.16») 


Fig.  44.  Power  determination  of  width  of  pass  band 
of  a  frequency  characteristic. 

In  practice  it  is  frequently  convenient  to  measure  the  bandwidth  of  a  linear 
network  between  the  points  at  which  the  amplification  in  terms  of  power  is  equal  to 
one  half  of  its  maximum  value,  or  the  amplification  in  terms  of  voltage  constitutes 
0.7  of  the  maximum.  For  theoretical  investigations  it  is  more  convenient  to  employ 
the  power  determination  of  the  bandwidth  in  the  form  of  (6.15).  However,  both 
methods  give  close  results  in  the  most  important  practical  cases. 

The  correlation  time  of  white  noise  which  has  passed  through  a  linear  network 
m-jy  be  determined  in  the  following  manner  ^compare  (5.32)|  ; 


to==- 


fo(T)rft 

.0 _ F,CU0) 

B{0)  2B(0)  ’ 


(6.17) 


or,  taking  into  account  (6.16*),  z  ~  I —  .  In  this  manner,  the  correlation  tine 

has  an  order  of  magnitude  Inverse  to  the  bandwidth  of  the  linear  network. 

Let  us  examine  several  examples  of  the  passage  of  white  noise  through  linear 
networks  with  frequency  characteristics  of  a  definite  fora. 

a)  The  passage  of  white  noise  through  an  ideal  linear  network  (filter  or  ampli¬ 
fier)  with  a  pass  band  equal  to  (Fig.  45,  a). 
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Fig.  45.  a)  Frequency  characteristic  of  ideal  linear  network. 

b)  Correlation  function  of  white  noise  at  output  net¬ 
work. 


The  equation  for  the  frequency  characteristic  of  an  ideal  linear  network  has  the 


form  of 


jCfl,  !•  — 

;  0, 


and,  consequently,  the  power  spectrum  of  noise  at  the  output  of  this  filter  is  equal 


to 


0.  I— 


When  A  ,  the  noise  at  the  output  of  an  ideal  linear  net^rk  represents 

an  example  of  a  process  whose  power  spectrum  lies  in  a  narrow  frequency  band  about 

a  high  frequency  to  ,  and  is  symmetrical  with  respect  to  the  latter.  Then  in  ac- 

o 

cordance  with  (6.14),  the  correlation  function  of  the  noise  at  the  output  of  an  ideal 
linear  network  is  equal  to 

-  tX 

F  ^  F n 

B{“)=-~cos «q1  j  Cp  cos vFzdio  =  - - cos  Wpt. 

0 

The  full  power  of  the  noise  is 
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and,  consequently,  the  expression  for  the  correlation  coefficient  of  the  noise  at 


the  output  of  an  ideal  filter  (amplifier)  has  the  form  of 


, 


^(')  =  -jj-cos»ot. 


(6.18) 


In  Figure  ^»’5l>,the  curve  of  correlation  function  (6,1 8)  is  indicated  by  the  dot¬ 


ted  line.  The  curve  is  plotted  only  for  the  positive  argument  of  “f  ;  in  virtue  of 


the  evenness  of  the  correlation  coefficient  the  ordinate  axis  is  its  axis  of  symmetry. 


The  solid  cur/e  corresponds  to  the  limiting  case  of  =  0,  i.e,,  to  the  correlation 


function  of  noise  at  the  output  of  a  low-frequency  filter  (amplifier)  with  a  band¬ 


width  of-^. 


If  the  correlation  time  is  determined  according  to  inequality  (5*32*),  it  fol¬ 


lows  from  (6, 18)  that  the  instantaneous  amplitudes  of  the  noise,  separated  by  a  time 


interval  of 


A  ii/  * 


may  be  considered  as  being  practically  independent  (cf,  p,  18o). 


If,  however,  the  correlation  time  Is  computed  on  the  basis  of  (6.17),  then 


a  _  *  ~L 

^o~J  a  ~24r 


Let  us  also  find  the  dispersion  cr  ^  of  the  derivative  of  white  noise  which  has 


passed  through  an  ideal  network  with  a  resonance  frequency  of  a)  and  a  bandwidth  of 


A  .  In  accordance  with  (5»6^*),o^j  =  — B**(0). 


For  the  computation  of  B"(0)  it  is  simplest  of  all  to  expand  B(jf)  into  a  series 


in  terns  of  f  ,  restricting  one’s  self  to  terms  of  no  higher  than  the  second  power 


B(t)=B(0)  ^  cos  B(0)  (•  -  tS  +•  •  •)  >^ 

.  < 

■  x(l-$ +...)=S(0)(l--^-1^+...). 
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Then 


“■  “  “  ®  <">  (h + ^  "  (tt  +  -0  • 


b)  Passage  of  white  noise  through  a  linear  network  (multistage  amplifier)  with 
a  frequency  characteristic  in  the  form  of  a  gaussian  curve  (Fig.  46, a) 


C(«)=Coe 


■  V" 


Fig.  46.  a)  Frequency  characteristic  in  the  form  of  gaussian  curve. 

b)  Correlation  function  of  white  noise  at  output  of  net  ork. 

The  power  spectrum  of  the  noise  at  the  output  of  such  a  network  also  has  the 
form  of  a  gaussian  curve 


The  correlation  function  of  the  noise  is,  in  accordance  with  (6.14),  equal  to 


■I 


—  cos»oX  I  e  '  cosTdi 


•* 

>• 


/■jp*  —  ~ 


Pe  *  cos«oT. 


The  full  power  of  the  noise  is 
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ftnd|  ccnsequently •  the  expression  for  the  correlation  coefficient  of  In  j 

ease  at  l^d  has  the  form  of 


(6.19) 


R{‘t)=t  cos«#o^, 

The  curve  of  correlation  function  (6.19)  is  shown  in  Figure  46, b.  Employing 
formula  (6.16)  we  find,  for  the  case  under  consideration,  the  power  pass  bandwidth 

p\/^ 


Li  this  manner,  the  parameter  p  in  the  equation  of  the  frequency  characteristic 


V 

coincides,  with  an  accuracy  of  constant  factor  ,  with  the  pass  bandwidth. 

The  dispersion  of  the  derivative  of  white  noise,  which  has  passed  through  a 
linear  network  with  a  gaussian  frequency  characteristic,  is  equal  to 

,  A 

Linear  networks  with  frequency  characteristics  of  the  two  types  examined  above 
are  an  idealization,  and  are  physically  unfeasible.  It  is  possible  to  examine  physi 
cally  feasible  linear  networks,  whose  frequency  characteristics  approach  those  ex¬ 
amined  above.  Some  standard  examples  of  such  frequency  characteristics  are  cited  in 


W- 


c)  Passage  of  >diite  noise  through  an  oscillating  circuit  formed  by  the  parallel 
connection  of  inductance  L,  resistance  R,  and  capacitance  C. 

As  is  known,  the  frequency  characteristic  of  the  oscillating  circuit  under  ex¬ 
amination  has  (Fig.  47, a)  the  form  of 


C(.)= 


where 


«  = 


2RC 


I 

Vtc 


•*0  —  =7= 


The  power  noise  spectrum  at  the  output  of  this  oscillating  circuit  is  defined  by 
the  function 


+  <«*<«* 
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t’^4‘ 


white  noise  which  has  passed  through  an  oscillating  circuit,  do  not  differ  in  type 
from  the  corresponding  curves  for  a  generalised  telegraph  signal  (cf.  Fig.  41). 

With  an  arbitral^  relationship  between  (J^and  ot  ,  formula  (6.14)  ceases  to  be 
valid,  and  to  find  a  correlation  function  it  is  necessary  to  compute  (for  example  - 
with  the  aid  of  residues  theory)  the  integral 


f  coil 

0 

As  a  result  of  the  computation  of  the  indicated  integral,  there  arc  obtained  the 
following  formulas  for  the  correlation  coefficient  of  noise  at  the  output  of  a 
circuit: 

^ihen  Wj  ~  0  (oscillating  circuit) 


cos«*ttfu 


/?  (t) = e“*  ^cos  «,  T -f  ^  ssn  u>,  I  ■:  1^  ; 

when  =  «  (boundary  of  oscillating  and  aperiodic  circuits) 

lAen  |3  ^  ®  (aperiodic  circuit) 

« (^) = 7  [  (‘  4-  j)  ( I  -  f  )  e-'’  I . 


(6.21) 


(6.22) 


(6.23) 


For  an  arbitrary  linear  network,  the  correlation  function  of  noise  at  the  out¬ 
put  of  this  ^stem  will  consist  of  a  sum  of  terms  of  the  type  of  (6.21)  and  (6.23), 
the  total  number  of  which  is  equal  to  half  the  total  number  of  poles  of  the  transfer 
function  of  the  linear  circuit  under  examination, 

4,  Optimum  Linear  Networks. 


A  problem  of  practical  importance  is  the  determination  of  such' a  linear  network 
(filter),  which  in  the  best  possible  manner  separates  a  signal  from  noise.  In  dis¬ 
tinction  from  Siict*  Ch«'  we  shall  examine  a  more  general  problem,  when  the  signal 
6  (t)  and  noise  (t)  distorting  it,  are  independent  random  stationary  functions  of 

time,  the  correlation  functions  and  power  spectra  of  which  are  known  and  equal  B  (tr), 

c 
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i*'>‘^S'*-»  •-=ir«'^.-’ 


FcCw),  and  B^(r),  F„(w)  respectively. 

The  sum  of  ?  (t)  +  ^(t)  is  fed  into  the  input  of  a  linear  network,  and  at  its 
output  there  is  received  the  random  process  C  (t)*  The  optimum  separation  of  signal 
from  error  is  given  by  a  linear  network  which  realizes  the  minimum  mean-square 
deviation  in  time  of  the  process  at  the  output  of  the  linear  network  from  the  signal 

^  (t),  i.e. ,  the  magnitude 

'  r‘-  .  ■ 


— r 


(6.24) 


The  possible  time  shift,  t^>0,  is  examined  in  problems  dealing  with  the  linear  pre¬ 
diction  of  a  signal. 

The  definition  of  the  optimum  criterion  is,  of  course,  conditional  and  depends 
on  the  methods  of  observing  the  output  process.  There  can  be  problems  where  the 
optimum  signal  separation  must  satisfy  a  condition  different  from  that  of  (6.24). 

The  indicated  criterion  is  usually  adopted  for  automatic  control  systems. 

Bnploying  (6,3)«  it  is  possible  to  represent  the  expression  for  mean-square 
error  in  the  form  of* 

+  (6.25) 

— r  o  ^  * 

Taking  into  account  the  independence  of  ^  (t)  and  ^(t),  and  also  bearing  in  mind  that 

.  r  '  -  • 

T 

')lA(-)rf*|  <//==  ' 

-r ' 0  ^  ' 


*  It  is,  consequently,  assumed  that  the  action  of  the  process  on  the  input  of  the 
filter  begins  in  distant  instants  of  time,  and  that  the  process  at  the  output  is 
also  stationary. 
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r  asos 

^riiS,  j  j'Jl'(<-«)+^*(/--«)l  ^ 

X[i(t  —  o)-\rti{t~v)\h{u)h{v)dudodt  = 


we  find 


=j||B.(« 


—  o)  4"  B,  («  —  t»)l  ft  («)  h{v)du  dv. 


m 

«  «o)  =  B,  (0)  -  2  J  B^  (/o  + 1)  h  (T)  dt  + 

00  00 

4*  J  ^  (B^  (a  —  o)  4-  B,  («  —  »)J  A  (u)  h  (w)  du  dv. 


(6.26) 


Thus  the  mean-square  error  in  the  reproduction  of  a  signal  at  the  output  of  a 
linear  network  depends  only  on  the  mean  power  characteristics  of  the  signal  and  noise 


i.e.,  on  their  correlation  functions.  The  various  signals  $  (t),  which  have  the 

K 


same  correlation  functions,  will  be  separated  from  the  noise  7?  (t),  likewise  with 

‘K 

uniform  correlation  functions,  in  the  best  manner  by  one  and  the  same  linear  network. 

We  shall  show  that  with  8^(2')  and  given,  the  best  separation  of  signal 

from  noise  will  be  accomplished  by  such  a  linear  network,  the  pulse  function,  h*(t), 
of  which  satisfies  the  integral  equation 


go 

(')  =  J  (Bj  (*  - «)  4-  B„  (t  - . «) J  /i*  («)  du  . 


(6.27) 


Vfe  shall  substitute  (6,27)  into  (6.26)  and  prove,  that  the  error  of  reproduction  is 
at  a  minimum  only  under  the  condition  that  h(t)  =  h*(t).  In  actual  fact,  as  a  result 
of  such  a  substitution  we  have 


00  00 


• = B,  (0)  ^2jj  IB,  (t  -  H)  4  B,  (t  -  «)1  A*  (tf)  A  (t)  du  dx  + 
00  00 

+  JJiB,(m  --»)4’B,(«  -•r)lA(M)A(u)d(id0  = 


esoa 


r 


=  B,  (0)  —  J  J  (B,  (t — If)  4-  B,  (t  —  «)]  A*  (//)  A  (t)  du  dx  — 
00  00 

— ]’  I  l^c  (^  ~«)  4-  -  tf)l  h  (tt)  A’(t)d«  dx  4 
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c  i-5 


w  «« 

+J  — -~v)\h(u)h(v)dudv  = 

«0  CO 

=  «e(0)- j  -«)4-B.(t-«)IA*(H)/i*(,)</ttrfx  + 

0  0 

00  00  .  ^ 

+ J  J[Be  («  -  (h-p)I  [/1  (tt)  -  -A*  (ff)l  [h  (v)~  A>)1  du  do. 

Baploying  (6.8*),  it  is  possible  to  rewrite  the  obtained  expression  in  the  form  of 


.  =  (0)  -  ^  J IF^  (^)  -I-  F^  (.)!  [  ^ 

0 
f 

■^2kJ  a(/») — A*(/ui)i-’rfw, 


(6.28) 


where  k(io)  and  k*(i«)  are  transfer  functions  of  linear  networks,  corresponding  to 
the  pulse  functions  h(t)  and  h*(t).  Since  only  the  third  terra  in  (6.28)  contains 
k(iu)  and  since  it  is  essentially  positive,  therefore  the  minimuni  value  of  €  will 
correspond  to  such  a  linear  network,  the  transfer  function  k(i«)  of  which  turns  the 
third  term  in  (6.28)  to  zero.  As  is  not  difficult  to  see,  this  will  take  place  under 
the  condition  that  k(iu)  H  k*(i(J),  Q.E.O. 

It  follows  from  (6.28),  that  the  minimum  value  of  the  mean-square  error  of 
signal  reproduction  at  the  output  of  an  optimum  linear  network  is  equal  to 


OC 

= Be  -i  J  H  -t-  HI  i  **('«>)  i-  = 

o*  ■ 

(^cH  •  (‘")  -r /",('»)! I *V'*)r I 


(6.29) 


The  fundamental  difficulty  in  solving  the  integral  equation  (6.27)  is  linked 
with  the  fact  that  Ainction  h*(t)  must  satisfy  the  condition  of  physical  feasibility 
(cf,  p. 217  )#  *3  3  result  of  which  the  lower  limit  of  integration  is  equal  to  zero, 
and  not  to—a>.  If  the  condition  of  pi^sical  feasibility  is  discarded  and  the  in¬ 
tegration  is  extended  from  -q^to  +  oo .  then  the  integral  equation  is  easily  solved 
by  means  of  a  Fourier  transformation  of  both  parts  of  (6.2?).  As  the  result  of  such 
a  transformation  we  obtain  a  solution  in  the  form  of 
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from  Which  ve  find  the  expression  for  the  frequency  characteristic  of  the  optimum 


linear  network* 


(6.30) 


The  mean  quadratic  error  in  the  reproduction  of  a  signal  at  the  output  of  a  linear 


network  is,  in  accordance  with  (6.29),  equal  in  this  case  to 


(6.3t) 


Formula  (6.31)  Indicates  that  an  error  in  the  reproduction  of  a  signal  in  the  optimum 
linear  network  may  be  made  equal  to  zero  only  ir»  the  case  when  the  spectra  of  the 
signal  and  noise  do  not  overlap,  l.e.,  when  F  («)•  F  contrary  case 

an  error  is  Inevitable. 

In  the  works  [2].  [3]  there  is  presented  a  complete  solution  of  the  problem  ex¬ 
amined  above,  with  allowance  for  the  physical  feasibility  of  linear  networks,  in¬ 
cluding  the  case  when  the  signal  and  noise  are  correlated. 

In  some  cases  the  amount  of  the  mean-square  error  e  ,  can  be  reduced,  if  the 

Diln 

pass  band  of  the  linear  network  varies  with  the  statistical  properties  of  the  input 
random  process  [12J. 

5.  Envelope  and  Phase  of  a  Random  Process. 

As  is  known  from  the  theory  of  the  Fourier  Integral  (cf. ,  for  instance,  E. 
Titchmarsh,  Vvedeniye  v  teoriyu  integrala  Fur*e.  Goskekhizdat,  1948[(i.e.,  Intro¬ 
duction, to  the  Theory  of  the  Fourier  Integral  (2nd  edition,  New  York,  1948)\),  by 
complying  with  certain  conditions,  the  function  S(t)  may  be  presented  in  the  form  of 
S(t)  =  a(t)  cos  y  (t). 

Functions  a(t)  and  y  (t)  are  determined  on  the  basis  of  the  formulas 
o(/)==/S?(0  +  o'5(0;  <?(/)  =  arc 

*  The  phase  characteristic  in  such  a  solution  does  not  depend  on  frequency,  and 
constitutes  a  constant  which  is  a  multiple  of  2rt, 
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vhere  o' (t)  is  a  so-called  conjugate  function  of  S(t),  which  is  uniquely  defined  on 
the  basis  of  S(t)  by  the  Gilbert  integral  transformation 


.(0  =  -i  ( 


Analogously,  with  several  very  general  assumptions  it  is  possible,  on  the  basis  of  a 
given  random  stationary  process  5  (t)»  to  form  by  means  of  a  Gilbert  transformation 
the  conjugate  of  ^  (t),  the  new  stationary  randoflt  process  (t) 


-r 


(6.32) 

Then  the  initial  random  process  5(t)  may  be  represented  in  the  form  of 

S(/)=£(/)cos«P(0.  (6.33) 

awhile  the  random  processes  E(t)  and  ^(t),  being  expressed  in  terms  of  ^  (t)  and  of  its! 
conjugate 7^  (t),  in  the  formulas 

£(0=/r(/)+T,*(0.  (6.34) 

(6.35) 


'‘•(0  =  arctgMg- 


The  random  processes  E(t)  and  defined  in  this  manner,  are  called  respectively 

the  envelope  and  the  phase  of  the  random  process  ^(t). 

Let  us  note  that  from  (6.3^)  it  follows  that  E(t)  >  ?(t),  i.e.,  the  random 
function  €(t)  nowhere  intersects  the  random  function  E(t).  In  addition 

and  therefore  at  points  where  $ (t)  =  B(t),  (i.e.,  7^  (t)  =  0),  there  holds  true 
—  =:  ~  ^  .  Thus  the  random  function  §  (t)  does  not  intersect  E(t),  and  at 
points  of  contact  has  a  common  tangent.  The  indicated  properties  explain  the  sense 
of  the  accepted  designation  of  random  function  E(t)  as  the  envelope  of  §(t). 

When  the  random  process  ^  (t)  is  represented  in  the  form  of  (6.33).  it  may  be 
regarded  as  a  harmonic  vibration,  modulated  with  respect  to  amplitude  and  phase  by 
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of  E(t)  are  strongly  correlated  and  its  correlation  function  varies  slowly  with  re¬ 
spect  to  T  ,  and  its  power  spectrum  will  be  concentrated  principally  in  the  low- 
frequency  range.  If  is  the  central  frequency  with  respect  to  which  the  narrow- 
band  spectrum  $  (t)  is  situated,  then  the  phase  (j>  (t)  is  equal  to 

‘t(0  =  “»o^  — 9(0. 


(6.36) 


the  random  function  (t)  also  turning  out  to  vary  slowly  in  time. 

Substituting  (6.36)  into  (6.33).  w®  obtain  the  following  representation  of  the 
narrow-band  random  process j 

?(/)  =  £  (/)  cos  <?  {/)  cos  (Dp  /  +  £  (0  sin  <f  (0  sin  ®q  /, 


(6.37) 


and  designating 


A(t)  =  E (1)  cos  <?  (/),  C  (/)  =  £  (t)  siatf  (/), 


(6.38) 


we  obtain 


« (0  =  >4  (0  cos  ®o  /  4-  C  (0  sin  ®o  t. 


(6.39) 


Thus  the  nanow-band  random  process  is  diaracterized  as  a  high-frequer.cy  oscil¬ 


lation  with  a  carrier  frequency  of  6>^  and  with  slowly  varying  envelope  and  phase. 


the  random  functions  S(t)  and  ^(t). 

The  possibility  of  such  a  representation  imposes  no  essential  restrictions  on 
the  power  spectrum  of  tR  process.  However,  the  representation  in  question  assumes 
the  greatest  practical  and  graphic  interest  for  narrow-band  processes  (cf.  Section  10, 
Chapter  V).  With  a  small  width  of  the  power  spectrum  of  process  §  (t),  its  envelope 
S(t)  varies  relatively  slowly  in  time  (as  compared  with  cos  ^  (t)),  i.e.,  the  values 


I 


.It 


<1 


It  is  possible  to  show  t^3j»  that  the  correlation  function  and  power  spectrum  of 

the  random  process  7)  (t),  conjugate  of  ^(t),  coincides  with  the  correlation  function, 

% 

B(?r)  and  the  power  spectrum  F(zr)  of  the  random  process  ^  (t),  and  that  the  mutual 
correlation  functions  of  these  processes  are  obtained  from  the  relationship 
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(6,i*0) 


Since 


X  {«(Oii(/+.t)}  =-m,  {tj(/)5(/  +  t)}  = 

m 

=  J  F  (w)  sin  o«  do». 

T,  (/)  =  £  (/)  sin  ‘I*  (/) = A  (/)  sin  /  —  C  (/)  cos  / , 


it  follows  from  (6.39)  and  (6.41)  that 


^(0=8  (0  cos  »»o  /  4-  ■»!  (0  sin  tuj  /, 
C  (/) = 5  (/)  sin  ti)j  /  ,j  (t)  cos  u)o  /. 


(6.41 ) 


(6.42) 

(6.43) 


Let  us  designate  by  M,  and  BQ^(t')  the  correlation  and  mutual  cor¬ 

relation  function  of  the  random  processes  A(t)  and  C(t).  Then  from  (6.4?)  it  follows 
that  ^ 

W = wi,  (/)  5  (/  -|- 1) J  cos  u>o  ^  4" 

4-"«j{5(0’i(/4-t)}sinu)ot, 

^Ac  (^)  *  ^CA  (^)  ==  {5  (0  4"')}  sin  Wp  t  — 

—  /«!  {  5  (/)  (/  4- 1)}  COS  0)p  t, 

.y' 

or,  expressing  the  correlation  and  mutual  correlation  functions  of  process  ^(t)  and 
7^  (t)  in  terms  of  the  power  spectimm  F(w)  of  process  ^(t),  we  obtain 

C» 

^A  W  —  (') = ■^  i  (“*)  COS  (ti)  —  (Dj)  t du),  (6.44) 

i 

*  (6  45) 

B/c  (')  =  (x)  =  ~J  f  (u))  sin  (u)  —  wj)  1  diu. 

If  the  power  spectrum  F(4tf)  of  the  random  process  ^(t)  has  a’ narrow  band  and  is  sym¬ 
metrical  with  respect  to  the  central  frequency  (J  ,  then  from  (6.44)  and  (6.45)  it 

0 

follows  (cf«  Section  10., Chapter  V)  that 

C»  •  ' 

'  Bj^{x)  =  Bf.{x)  =  ~^f*{m)co$mxdia  =  a{x),  (6,46) 

«.cW=-«mW=o. 

where  a(r)  is  a  slowly  changing  function  of  argument  r  . 


(6.46) 

(6.47) 


From  (6.39)  and  (6.46)  it  follows  that  the  correlation  function  B(r')  of  the 
narrow-band  random  process  ^ (t)  is  equal  to 
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Tr^’~57STr:.-:;7r 


I 


B  (t) = m,  {I  (/)  £  (/  4- 1)|  =  a  (t)  cos  o)^  t. 


(6.48) 


which  coincides  with  (5.82),  Here,  however,  becomes  clear  the  physical  sense  of  the 
function  a(tr)  in  (5*82),  which  is  the  correlation  function  for  each  of  the  slowly 
varying  processes  A(t)  and  C(t),  tied  to  the  narrow-band  process  ^  (t)  by  the  re¬ 
lationship  (6.39). 

43  iraa.seen  from  Section  3,  white  noise,  which  has  passed  throTi^  a  naorrow-band.  litveer 
network  with  a  resonance  frequency  of  ,  constitutes  a  narrow-band  random  process. 

If,  besides  that,  white  noise  is  normal,  then  the  process  at  the  output  of  the 
narrow-band  linear  network  may  be  represented  as  a  sun  of  the  type  of  (6.39),  in 
which  A(t)  and  C(t)  will  be  incoherent  normal  and,  consequently,  independent  random 
processes, 

6,  Correlation  Function  and  Power  Spectrum  of  Random  Process  at  Output  of 

Honlinear  Network. 

.  __ii  Section  3  it  was  indicated'  that  in  linear  ^sterns  the  spectrum  and  correiation 
function  are  uniquely  determined  by  the  frequency  characteristic  of  the  network  and 
by  the  spectrum  of  the  process  at  its  input.  In  order  to  determine  the  power 
spectrum  of  a  random  process  or  of  its  correlation  function,  it  is  insufficient  to 
know  only  the  spectrum  (or  the  correlation  function)  at  its  input,  and  it  is  necessary 
also  to  have  the  expression  for  the  two-dimensional  distribution  function  of  the  in¬ 
put  random  process. 

Let  the  characteristic  of  a  nonlinear  and  noninertial  network  have  the  form  of 

•  y=!i»)  (6.49) 

and  let  there  be  known  the  second  distribution  function  *2'  ^  ^ 

stationary  random  process  ^  (t)  at  the  input  of  the  nonlinear  network.  Then,  em¬ 
ploying  the  rules  cited  in  Chapter  HI  for  finding  mean  values  of  the  functions  of 

I 

random  variables,  for  the  correlation  function  of  a  random  process  at  the  output  a  j 
nonlinear  network  we  obtain  the  following  expression;  | 
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«  0» 

—  j  J  /  (jf  l)  /  U2)  Wi(x,,  Xj,  t)  dx,  dXj. 

— OD 

Ihe  mean  value  of  «*.  (-the  direct  component),  is,  in  accordance  with  (5.23)»  de¬ 
termined  on  the  basis  of  the  asymptotic  value  of  B(r),  i.e. , 

^  ~  J  S  n»m  “^2  (^1.  Xa,  t)J  dxi  dxa.  (6. 51 ) 

— »  —00  *■**• 

If  a  purely  random  process  is  being  examined,  then  when  — *00  the  values  of 

^  (t)  and  §  (t  +Zf  )  become  independent  and  the  two-dimensional  distribution  function 
becomes  in  this  case  equal  to  the  product  of  two  one-dimensional  distribution 
functions.  “Ihen  from  (6.51 )»  after  separating  the  variables  of  integration,  we  ob¬ 
tain 

°*=f  J  f^x)w^{x)dx]. 

J  (6.52) 


3 

if 

I 

I 


Having  found  the  correlation  function  of  a  random  process  at  the  output  of  a 
nonlinear  network,  it  is  possible,  employing  (5*^).  i.e.,  effecting  a  Fourier 
transformation,  to  obtain  the  power  spectrum  of  this  process. 

If  the  process  at  the  input  of  a  nonlinear  network  constitutes  a  sum  of  a 
stationary  random  process  and  a  determined  process,  then  contrary  to  the  case  of  a 
linear  network,  the  passage  through  the  network  of  the  determined  part  may  not  be 
treated  separately  from  the  purely  random  process.  The  passage  through  the  nonlinear  | 
network  of  both  terns  must  be  studied  jointly,  /I 

j 

The  direct  computation  of  the  integral  in  formula  (6. 50)  is,  as  a  rule,  very  ’ 

difficult.  Therefore  it  is  expedient  first  to  transform  it  into  a  form  which  sepa-  ,i 

rates  the  variables  of  integration  in  the  double  integral.  Several  methods  exist  for 
such  a  simplification  of  formula  (6.5O). 

A.  Series  expansion  of  two-dimensional  probability  density. 

Let  W|^(x^)  and  one-dimensional  distribution  functions,  corresponding; 
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to  a  two-dimensional  probability  density  of  WgCx^,  Xg,  *'),  i.e., 

00 

»u(*i)=  J  a>2(Jfi.  *2. ‘)rfx,.. 


(6,53) 


»;2(jf2)=/»2(jf|.  Xj,  t)rfx,. 

—00 

I^t  US  take  and  weighting  functions  and  let  us  construct  two  ag¬ 

gregates  of  nomalited  orthogonal  polyriomials*-  Q  ,(x*)  and  Q  -(x*),  which  must 

nl  1  D2  z 

satisfy  the  conditions  of  orthogonality 

J  ,,  y,, 

\Om:y^n.  (o,5V 

jj»i2 (Jf2)  QJXi)  Q„,  (Xj)  dx,  =  ^  ^  (6.54*) 

Then  the  two-dimensional  distribution  function  W2(x^,  X2,  r)  nay  be  expanded  into  a 
double  series  in  terns  of  these  orthogonal  polynomials 


(6.54) 

(6.54*) 


«»2(X,.  X, (X,)a',2(x,);^  f  W  Qm,  (^l)  Q„2  (^*)- 


m-On-0 


(6.55) 


The  coefficients  can  be  found  by  multiplying  both  sides  of  equation  (6.55)  by 
QkYtx^r  Qj2(^2^  and  integrating  with  the  use  of  the  orthogonality  condition  of  (6.54) 
and  (6.54*).  As  a  result,  all  terms  vanish,  except  one  (k  =  m,  1  »  n),  and  conse¬ 


quently 


0» 

.(•)=  J  J  w,(x„  x„  vQ„,(x,)Q^(x,)dx,(/xj 


(6,56) 


In  many  cases  of  practical,  importance  a  =  0  when  min.  For  this  class  of 

SIfl  I 

distribution  functions  formulas  (6.55)  and  (6.56)  become  simpler 


W2  (•'f I,  Xi,  ■:)  =  a?,,  (.V,)  (x,)  V  a ,  (t)  (;c,)  .  (x,), 

00  00 

«.(•)=  j‘  J*  »,(X,.  X,.  t)Q,„  (.v,)Q„,(x,)dAV/x,. 


(6.57) 


(6.58) 


Substituting  (6.57)  into  (6,50)  and  separating  the  variables  of  integration,  we 
obtain 

*  On  the  construction  of  an  aggregate  of  normalized  orthogonal  polynomials  on  the 
basis  of  an  assigned  weighting  function  of,  in  the  book  of  V.  L.  Gonchareov  "Teoriya 
interpolirovaniya  i  problizheniya  funktsiy"  ("The  Theory  of  Interpolation  and 
Approximation  of  Functions"),  Costekhizdat,  1954. 
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i 


where 


5  (-)==  2  <•,„<•;„«„  (t). 

A-O 

00 

^IH=  J  /(■»C|)Q«|(^»)«'ll('fl)rf'fl. 

—30 
‘  00 

^5«  ~  J'/  (•'^2)  Q„;  (•'^2)  "^12  (-^2)  t^Xj. 


(6.59) 


(6.60) 


(6.61) 

Formula  (6.59)  yields  the  expression  of  the  correlation  function  of  a  stationary 
random  process  at  the  output  of  a  nonlinear  network  in  the  form  of  the  series  of 
functions  SL^^Cr)#  which  are  determined  by  the  correlational  characteristics  of  the 
process  at  the  input.  Such  a  method  of  computing  B(v)  may  be  called  the  direct 
method  (or  the  method  of  correlations). 

If  the  process  at  the  input  is  nonstationary,  the  two-dimensional  probability 
density  W2  will  be  also  a  function  of  time  t  and  therefore  c^^^,  C2jj  and  a^  will  also 
depend  on  t.  For  determination  of  the  correlation  function,  a  supplemental  time 
averaging  will  be  necessary. 

6.  Contour  integral  method. 

The  second  method  consists  in  making  use  of  the  fact  that  the  characteristics 
of  several  nonlinear  networks  permit  representation  by  means  of  a  contour  integral 
in  the  form  of 


(6.62) 


If  (6,62)  is  substituted  into  (6.50),  by  changing  the  order  of  integration  we  find 

®  (■) = IS"  J  ^  ('"»)  J  ^  ('"2)  X 

«e  OB  ** 


and,  employing  (3.7^).  we  obtain 

^ ^ ~  lir  0'"i)  J ^ ('"2)  («i.  ff-j-  ') 


(6.63) 
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where  OgCu^,  Ug,  r  )  is  the  two-dimensional  characteristic  function  of  a  random 


process  at  the  input  of  a  nonlinear  network.  The  function  '^2‘  ^  ^ 

analogously  to  (6.57)  b®  expanded  into  the  series 


®a(^i*  ^a*  “)  —  ®ii (^i)‘®ia(^^2)^  Affli(Wi) Af„2(«2)6^j(':), 


(6.64) 


where  and  polynomials  of  the  n-th  power. 

Then  from  (6.63)  we  obtain  the  expression,  analogous  to  (6.59) 


(6.65) 


in  which- 


*C| 

~  2*  J  « (‘“a)  (Mj)  012  {Ih)  dit^. 


(6.66) 


(6.67) 


If  the  process  at  the  input  is  nonstationary,  then  the  two-dimensional  charac¬ 


teristic  function  will  be  also  a  function  of  time  t,  and  therefore  d.  ,  d.  »  and 

*  in  zn 


will  also  depend  on  t.  For  determination  of  the  correlation  function,  a  sup¬ 


plemental  tine  averaging  will  be  necessary. 


C.  The  envelope  method. 


In  problems  where  a  narrow-band  process  is  subjected  to  a  nonlinear  transforma¬ 


tion,  it  is  expedient  to  employ  the  representation  of  that  process  in  the  form  0/ 


(6.33) t  taking  into  account  (6,36) 

t(/)  =  £(0 cos (V  -9(01. 


and  to  represent  the  process  at  the  output  of  the  nonlinear  network  in  the  form  of 


ti(o=n?(oi=/o(£)+ 

+  /|  (£)  cos  O'ly/  —  <P)  /j  (£)  cos  2  (u»o/  —  9)  -f- . 


(6.68) 


where 


(l^)  =  ~  J  /(£cos  •;.)  cos 


(*o=L'V:**=2,  n^O). 


(6.68») 
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It  can  be  shown*  that  E  and  y  a^'®  independent,  and  the  spectra  f^(E)  practically  do 
not  overlap.  The  correlation  function  of  the  process  7^(t)=:  f  Ml  is  repre¬ 
sented  in  this  case  by  the  series 


fl(‘)  =  ^B„(-)cos/iu)oT, 


A«0 


(6.69) 


in  which  the  functions  Bjj(tr)  are  determined  by  the  formula 


dooo 


(6.70) 


where  Wp(y^,  y^,  2*)  is  the  two-dimensional  distribution  function  of  the  envelope  of 
the  process.  The  envelope  method  was  developed  by  V.  I.  Bunimovich  [13]. 

7.  Distribution  Fun.^-tion  of  Random  Process  at  Gutnut  of  Nonlinear  Network, 

As  was  already  noted  in  Section  1,  for  a  nonlinear  and  noninertial  network  the 
function  f(>:)  in  (6,U9)  is  singled-valued,  i.e,,the  value  of  the  random  function "^Ct), 
characterizing  the  process  at  the  output  of  a  nonlinear  network  at  any  instant  in  time,  is 
detemined  only  by  the  value  § (t)  of  the  random  function  corresponding  to  the  input 
at  the  same  instant  in  times  nCt)  =  f  j  .  Therefore  for  determination  of  the 

distribution  function  w^(y^»  y2****y,-j«  of  the  random  process  at  the  output 

of  a  nonlinear  network,  it  is  sufficient  to  have  the  corresponding  distribution 
function  •ir^^(x^ ,  X2*...Xj^,  t^,...tj^)  for  the  input,  and  to  employ  the  general  fonaula 
set  forth  in  Chapter  III  for  the  replacement  of  the  variables  in  distribution 
functions  in  the  transformation 

2, (6,71) 

where  5*==«(^). 

Sometimes  these  computations  may  be  simplified,  if  first  are  found  not  the  dis¬ 
tribution  functions  themselves,  but  their  characteristic  functions. 

The  employment  of  characteristic  functions  for  determining  the  distribution 


*  For  pro^f  of  the  Independence  of  E  and  (j)  ,  it  is  sufficient  to  show  that  if 
and  ^  are  Independent,  and  normally  distributed  with  parameters  of  a,  =  ao  =  0, 
o'  « O'  «  O'  *  then  ^  Z~  independent. 
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functions  of  a  process  at  the  output  of  a  nonlinear  network  will  become  necessary, 
each  time  that  fxinction  f(x)  in  (6.70)  does  not  have  a  reciprocal  (e.g. ,  in  the 
case  of  half-wave  detection)  and  the  employment  of  the  formulas  in  Chapter  III  is  im¬ 
possible. 

Let  us  employ  (3*73)  and  write  the  n-dimenslonal  characteristic  function  of  a  process 
at  the  output  of  a  nonlinear  network 


or 


e^(o,,o,....o^, gr= 

«  /  J  •••'  <'•>  W  (W  f  • .  -f  V  (trt)l  I 

s=m,^e 


(6.72) 


M  «B 


Jl«J  (*1>  K./  (*.)  1-  .  .  t (*„>1 


=  J  . . 

“<#—09 

'  X  (■^i»  JCj. . . .  fj, . . .  i^)‘dx\dx2.  • .  dx^. 


(6.73) 


Performing  an  inverse  Fourier  transformation  upon  0^,  we  find  the  n-dimenslonal 
distribution  function  W^(y^ #  y2’***^n'  ^1’  ^  process  at  the  output  of  a 

nonlinear  network. 

For  determining  the  statistical  characteristics  of  the  envelope  E(t)  and  phase 
(t)  of  the  narrow-band  random  process  ^  (t),  we  represent  it  in  the  form  of  a  sum 
(cf.  p.  237) 

?(/)  =  £  (0  cos  (uV  -  «p  (01  =  (/)  cos  u)o/  +  C  (0  sin  wof, 

(6.74) 

where  A(t)  and  C(t)  are  independent  random  processes.  If  the  n-dimensional  dis¬ 
tribution  functions  of  these  random  processes  are  equal  respectively  to  w^^(x^,  x^, 
...Xjj,  t^,...t^)  and  y2».*.yn»  t^,,..tj^),  then  ty  virtue  of  their  independ¬ 

ence  the  joint  distribution  of  these  processes  will  be  characterized  by  a  2n-dl- 
mensional  distribution  function 


®^2i,  (*»»  ••••*),*  j'l.  ••  -  ti,...  /„)  = 

~  “’#1  (Jf|*  •  •  •  . . .  /,)• (jfi, . . .  y„,  /|. . . .  /  ). 


(6.75) 


In  order  to  find  the  distribution  functions  of  the  envelope  E(t)  and  phase  (|)  (t), 
we  transform  (6.75)  to  polar  coordinates 

F-TS-'?8tl/V  245 


245 


THIS  PAGE  IS  BEST  QUALITY  PRA<JII,CAJaiI»B 

EBm  COPY  FURNISHED  TO  DDC  - - 

independent,  the  characteristic  function  *2****  ’‘n* 

process  (t)  obtained  as  a  result  of  the  nonlinear  transformation  (6.49),  nay  in 
accordance  with  (3.73)  be  computed  according  to  the  formula 


I 'times 

^ ^Jyu •  •  •  J',.  . QdXi . . . dxjyi ...dy^ 


(6.81) 


8.  On  the  Determination  of  the  Distribution  Function  of  a  Random  Process 
at  the  Output  of  a  Linear  Network 

As  we  have  seen  in  |2,  the  problem  of  the  transformation  of  the  correlation 
function  of  a  process  and  its  spectrum  is  sufficiently  simple  in  a  linear  network  and 
does  not  even  require  a  knowledge  of  the  distribution  functions  of  the  input  process. 

An  incomparably  more  complicated  problem  is  the  determination  of  the  distribution 
function  of  process  output  of  the  linear  network. 

Only  in  one  special  case,  when  the  process  $^^(t)  at  the  input  of  the  linear 
network  is  normal,  can  this  problem  be  solved  simply.  Tne  random  process  ^  (t)  is 

H 

the  limit  of  the  integral  sum  where 

. ..  *-0 

when  (6.3)].  The  random  variables  (t),  ^  (t  -t^),...^^ 

(t  -  in  the  case  at  hand  are  linked  by  an  N  +  1  -dimensional  normal  law  of  dis¬ 
tribution.  Since  a  sura  of  normally  distributed  random  variables  (even  of  dependent 
onesj  cf«  Seetioji  9j  Ch«  3)  is  normally  distributed,  then,  consequently,  the  sum  in 
question  has  a' normal  distribution  for  any  II, 

Thus,  in  passage  through  linear  networks,  the  normal  random  process  retains  its 
distribution  functions,  i.e,,  remains  normal.  Changes  occur  only  in  the  correlation 

function  and  power  spectrum,  in  accordance  with  the  formulas  indicated  above. 

/ 

It  has  already  been  noted  in  Section  i,that  an.' analysis  typical  of.  radio  apparatto  is  ' 
one  of  a  linear  network  following  a  nonlinear  element  (Fig.  43).  Hierefore  even  in 
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those  cases  when  a  normal  random  process  is  subjected  to  a  nonlinear  transformation, 
this  process  at  the  Input  of  the  succeeding  linear  network  is  no  longer  normal. 

The  problem  of  the  transformation  of  a  distribution  function  in  a  linear  net¬ 
work,  at  the  input  of  which  there  acts  a  random  process  differing  from  the  normal,  is 


I 

*  -4 
(  is 


! 

I  ^ 


j  3 


an  extremely  difficult  one.  There  exist  several  approximate  methods  of  solving  this 


!  4 


-Si 


problem,  each  of  which  is  based  on  special  assumptions  with  respect  to  the  statistical 
characteristics  of  the  input  random  process  and  the  properties  of  the  linear  network 
itself  [15].. 

In  Chapter  II,  this  problem  will  be  examined  under  the  assumption  that  the 
process  at  the  input  of  the  linear  network  is  the  square  of  a  normal  random  process. 

Here  we  shall  pause  merely  for  one  general  approximate  method  of  determining  one¬ 
dimensional  statistical  characteristics,  which  consists  in  the  computation  of  the 
distribution  moments  m,^  of  a  process  at  the  output  of  a  linear  network  [9]>[l(^ .  Since  th|  J 
one-dimensional  characteristic  function  0  (v)  may  be  represented  by  a  series  [cf. 

0.«7)I 


i 


t  ? 

R 


therefore,  having  a  sufficient  number  of  distribution  moments,  it  is  possible  with  a 
certain  degree  of  approximation  also  to  synthesize  a  distribution  function  or,  at 
least,  to  have  a  conception  of  its  type. 

If  there  were  known  the  n-dimensional  correlation  function  of  the  random  process 
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«-+tW=  Jjc*+‘»,(x./)rfx  = 

— Oi 

=«i{^*^‘(/)}=lima,(t„...,^,/). 


(6.82) 


<i<«0 
•  •  •  • 


]i>t  U3  deaignata  by  F„(  ••<«>„)•  spectrum  (Fourier  transformation)  of  the 

n-dimenslonal  correlation  function  B*  (  ....  r^),  averaged  over  time.  Then 

<(^i.  ♦••%)= 


m  m 

If  C  e  i  \  •  •  *  +•«  j 

=inr3-J-‘" . “•*' 


.  dw_ 


— M  -« 

n  times 


(6.83) 


From  (6.82)  and  (6.83)  there  follows 

lime; 


«  00 


t|-.0 


(<«)'•  I  ‘  •  J  . .  ‘/‘"I  .  . .  rfu),. 


(6.84) 


n  times 


Thus,  the  n  +  1-th  initial  moment  of  the  one-dimensional  distribution  function 
of  a  stationary  process  (or  the  mean  Vilue  uf  this  mooaent  for  s  r.onj>tJtion?'.Ty  procs?-) 
.is  equal  to  the  full  volume  of  the  spectrum  of  the  time-averaged,  n-dimensional  cor¬ 
relation  function  of  this  process. 

Consequently,  In  order  to  determine  the  n  +  1-th  initial  moment  of  the  one¬ 
dimensional  distribution  function  of  a  process  at  the  output  of  a  linear  network  it 

is  sufficient  to  know  for  this  process  the  indicated  n-dimensional  spectrum  ?  (ui 

n  1  ’ 

t®  found  by  means  of  the  transformation  formula  for  such  spectra  ir 

linear  networks; 


'  •»-!  /  M-1 


(6.85) 


which  constitutes  a  generalization  of  formula  (6.4)  for  the  one-dimensional  case. 
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Here  k(iw)  is  the  transfer  function  of  the  linear  system. 

It  should  be  noted,  that  in  those  cases  when  the  spectrum  band  of  the  process 
at  the  input  of  a  linear  network  is  much  wider  than  the  pass  band  of  this  network, 
the  process  at  the  output  always  has  a  tendency  toward  normalization,  i.e. ,  its  dis¬ 
tribution  functions  are  sufficiently  closely  approximated  by  normal  ones.  This  sit¬ 
uation  is  a  direct  consequence  of  the  central  limit  theorem  of  lyapunov,  set  forth  in 
Section  1  of  Ch.  IV.  In  fact,  let  us  replace  the  process  at  the  input  of  a  linear 


network  by  a  sequence  of  base  pulses,  the  duration  iST  of  each  of  viiich  is  much  less 

I 


than  ”3^  ,  where  A f  is  the  pass  bandwidth  of  the  network.  The  process  at  the  output 
of  the  linear  network  after  the  passage  of  such  a  sequence  of  base  pulses  will  in 
practice  differ  little  from  the  process  which  takes  place,  when  at  the  Input  there 
acts  the  original  random  process.  The  reaction  of  the  linear  network  to  a  sequence 
of  base  pulses  may  be  represented  In  the  form  of  a  sum 


H  (/)  =  A  (<  -  AA?)  5  (AAI)  AT. 


(6.86) 


If  the  power  spectrum  of  the  initial  random  process  $(t)  is  so  wide,  that  the 


correlation  lime  of  this  process  is  much  smaller  than  AT,  then  ary  two  base- 


pulse  amplitudes  $  (K  AT)  and  ^(r  A  T  )  r)  vlll  be  independent.  It  can 
then  be  seen  from  (6.86),  that  the  process  at  the  output  of  a  linear  network  is  the 
sum  of  a  large  number  of  independent  random  variables,  which  in  accordance  with  the 
lyapunov  theorem  should  have  a  distribution  close  to  the  normal. 

S.nce  the  product  of  the  effective  width  of  the  spectnim  of  a  random  process  AF 


by  the  correlation  time  '^is  on  the  order  of  unity,  therefore  from  the  conditions 


cited  above  of  the  normalization  of  a  process  at  the  output  of  a  linear  netwoi'k 


it  follows,  that 


*  -  I  i/ 


(6.87) 
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1. e.,  for  the  normalization  of  a  process  it  is  necessary,  that  the  band  of  a  linear 
netvfork  be  much  narrower  than  the  effective  width  of  the  power  spectrum  of  the 
random  process  acting  on  the  input. 

Of  course,  the  considerations  cited  here  do  not  constitute  a  rigorous  proof  of 
the  theorem  of  the  normalization  of  a  process,  when  — ►  0.  For  such  a  proof  it 

would  be  necessary  to  take  into  account  the  presence  of  correlation  links  between 
the  terms  of  sum  (6.86),  and  to  expand  the  central  limit  theorem  to  the  sum  of  de¬ 
pendent  random  variables  (cf.  S.  N.  Bemshteyn.  Teoriya  veroyatnostey  (ProbubilS  ty 
theory),  4-th  edition.  Gostekhizdat,  1946.) 
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